Timing calibration apparatus and method for a memory device signaling system

ABSTRACT

A memory system includes a memory controller and a memory component coupled to each other. An interface of the memory component is configured to receive a first signal from the memory controller with read request information, retrieve the read data information from the memory core in response to the request information, and transmit to the memory controller a second signal containing the read data information. The read data information includes read data symbols, where the average duration of the read data symbols, measured at the interface, defines a symbol time interval. A first external access time is measured at the interface between a first read request and read data transmitted by the interface in response to the first read request. A second external access time interval is measured at the interface between a second read request and read data transmitted by the interface in response to the second read request. The difference between the first external access time and the second external access time is greater than one-half of the symbol time interval.

This application claims priority to U.S. provisional application No.60/343,905, filed on Oct. 22, 2001, and U.S. provisional application No.60/376,947, filed on Apr. 30, 2002, both of which are herebyincorporated by reference.

FIELD OF THE INVENTION

This invention generally relates to the field of digital circuits, andmore particularly to an apparatus and method for timing calibration andmemory device signaling systems.

BACKGROUND OF THE INVENTION

Integrated circuits connect to and communicate with each other,typically using a bus with address, data and control signals. Today'scomplex digital circuits contain storage devices, finite-state machines,and other such structures that control the movement of information byvarious clocking methods. Transferred signals must be properlysynchronized or linked so that information from a transmit point isproperly communicated to and received by a receive point in a circuit.

The term “signal” refers to a stream of information communicated betweentwo points within a system. For a digital signal, this informationconsists of a series of “symbols,” with each symbol driven for aninterval of time. In digital applications, the symbols are generallyreferred to as “bits” in which values are represented by “zero” and“one” symbols, although other symbol sets are possible. The valuescommonly used to represent the zero and one symbols are voltage levels,although other variations are possible.

In some cases, a signal will be given more than one name (using an indexnotation), with each index value representing the signal value presentat a particular point on a wire. The two or more signal names on a wirerepresent the same information, with one signal value being atime-shifted version of the other. The time-shifting is the result ofthe propagation of voltage and current waveforms on a physical wire.Using two or more signal names for the same signal information allowsfor easy accounting of the resulting propagation delays.

The term “wire” refers to the physical interconnection medium whichconnects two or more points within a system, and which serves as theconduit for the stream of information (the signal) communicated betweenthe points. For example, but without limitation, a wire can be a copperwire, a metal (eg., copper) trace on a printed circuit board, or a fiberoptic cable. A “bus” is a wire or a set of wires. These wires arecollected together because they may share the same physical topology, orbecause they have related timing behavior, or for some other reason. Theassignment of wires into a bus is often a notational convenience. Theterms “line,” “connection” and “interconnect” mean either a bus, wire orset of wires as appropriate to the context in which those terms areused.

The term “signal set” refers to one or more signals. Whenever a signalor signal set is described herein as being coupled to or attached to adevice or component, it is to be understood that the device or componentis coupled to a wire, set of wires or bus that carries the signal.

The mapping of a signal onto a physical wire involves tradeoffs relatedto system speed. The use of one physical wire per signal (single-endedsignaling) uses fewer wires. The use of two physical wires per signal(differential signaling) permits shorter bit intervals. The mapping ofsignals onto physical wires can also involve optimization related tosystem resources. Two different signals can share the same wire (i.e.,they are multiplexed) to minimize the number of physical wires.Typically, this must be done so that the potential timing conflicts thatresult are acceptable (in terms of system performance, for instance).

The interval of time during which a bit or symbol is transmitted orreceived at a particular point on a wire or at a device interface is the“symbol time interval,” “bit time,” “bit time interval,” “bit window,”or “bit eye.” These time interval terms for transmitting and receivingare used interchangeably. Usually, the bit interval for transmit signalsmust be greater than or equal to the bit interval for receive signals.

In FIG. 1, a bus 20 interconnects a memory controller 22 and memorycomponents (MEMS) 24. Physically, the bus 20 comprises traces on aprinted circuit board or wiring board, wires or cables and connectors.Each of these devices 22, 24 has a bus output driver or transmittercircuit 30 that interfaces with the bus 20 to drive data signals ontothe bus to send data to other integrated circuits. Each of these devicesalso has a receiver. In particular, the bus output drivers 30 in thememory controller 22 and MEMS 24 are used to transmit data over the bus20. The bus 20 transmits signals at a rate that is a function of manyfactors such as the system clock speed, the bus length, the amount ofcurrent that the output drivers can drive, the supply voltages, thespacing and width of the wires or traces making up the bus, and thephysical layout of the bus itself. Clock, or control, signals serve thepurpose of marking the passage of time, thereby controlling the transferof information from one storage location to another. The memorycontroller 22 is connected to a central processing unit (CPU) 40 andother system components 50, such as a graphics control unit, over bus45.

As signals pass over a bus and through device interfaces, the signalsexperience propagation delays. Propagation delays are affected byvariables such as temperature, supply voltage and process parameters(which determine physical characteristics of the devices sending andreceiving the signals). For example, at a low operating temperature witha high supply voltage, signals may be transmitted with a relativelyshort delay. Alternatively, at a low supply voltage and high operatingtemperature, a significantly longer delay may be experienced bytransmitted signals.

Variations in the process parameters, which result in variations in theperformance of otherwise identical devices, cause devices either on asingle bus, or devices on parallel buses to experience different signalpropagation delays. The load on each bus, which depends on the number ofdevices connected to the bus, may also affect signal propagation. Insum, the phase relationships between transmitted and received signals,are affected by numerous factors, some of which may change during theoperation of a system. Small changes in propagation delays can result indata transfer errors, especially in systems with very high bit (or moregenerally, symbol) transfer rates, and thus very short bit (or symbol)times. In order to account for actual propagation delays, it isdesirable, especially in systems with very high bit (or symbol) transferrates (e.g., without limitation, 250 Mb/s or higher) to synchronizesignal transmitters and receivers, to account for actual propagationdelays. The present invention provides systems and methods fordynamically synchronizing signal transmitters and receivers, even whenthe variations in propagation delays caused by temperature, voltage,process and loading variations exceed an average symbol time interval.Normally, a variation in propagation delay of even a half symbol timeinterval will cause a memory system or data transfer system to fail,because movement of a half symbol time will cause the data sample pointto move from the center of the data eye to the edge of the data eye.Change in the propagation delay of more than a half symbol time will, inconventional prior art systems, cause the wrong symbol to be sampled bythe receiving device. In the present invention, such changes inpropagation delay are automatically “calibrated out” by the use ofdynamic propagation delay calibration apparatus and methods.

SUMMARY OF THE INVENTION

This invention generally relates to apparatus and methods for timingcalibration and device signaling systems in which phase offsets ofsystem device components are not necessarily fixed during normal systemoperation, but are allowed to vary over a drift range. Systems designedin accordance with the present invention allow for the calibration andadjustment in memory device signaling systems so that the transmissionof signal information between a first device (such as for example amemory controller) and a second device (such as for example a memorycomponent) occurs without errors even when the accumulated delaysbetween the first device and second device change by a half symbol timeinterval or more during operation of the system. This invention can alsobe utilized for communications between circuit blocks within a singleintegrated circuit or component.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention andadvantages thereof, reference is now made to the following descriptiontaken in conjunction with the accompanying drawings:

FIG. 1 is a block diagram of a prior art bus connecting a memorycontroller and a number of memory components.

FIG. 2 is a block diagram of a prior art static mesochronous memorysystem connecting a controller and a memory.

FIG. 3 is a timing diagram for the signals of the static mesochronousmemory system in FIG. 2.

FIG. 4 is second example of a block diagram of a prior art staticmesochronous memory system connecting a controller and a memory.

FIG. 5 is a timing diagram for the signals of the static mesochronousmemory system in FIG. 4.

FIG. 6 is a block diagram of a dynamic mesochronous memory systemconnecting a controller and a memory in accordance with a preferredembodiment of the present invention.

FIG. 7 is a block diagram of a dynamic mesochronous memory system inaccordance with an alternate preferred embodiment of the presentinvention.

FIGS. 8A and 8B depict dynamic mesochronous memory systems in which theX and Y buses are parallel to each other.

FIG. 9 is block diagram of a dynamic mesochronous memory system inaccordance with an alternate preferred embodiment of the presentinvention.

FIG. 10 depicts sample splitting element configurations.

FIG. 11 depicts sample internal and external termination componentvariations.

FIG. 12 is logic diagram for a baseline system configuration 1200 for acomponent shown in the system topology diagrams.

FIG. 13 depicts a dynamic mesochronous memory system in accordance withan alternate preferred embodiment of the present invention.

FIGS. 14A and 14B depict a sequence of timing signals for a readtransfer of the system topology of FIG. 13.

FIG. 15 depicts a sequence of timing signals for a write transfer of thesystem topology of FIG. 13.

FIG. 16 is block diagram of a memory system in accordance with apreferred embodiment.

FIG. 17 is logic diagram for a memory system of the preferredembodiment.

FIG. 18 shows a sequence of timing signals for block M1 of the memorysystem of FIG. 16.

FIG. 19A is a logic diagram of the M2 module in the memory system ofFIG. 16 for transmitting read data on a bus.

FIG. 19B is a logic diagram of the M3 module in the memory system ofFIG. 16 for receiving write data.

FIG. 20 is a block diagram for a controller of the topology of FIG. 13comprising three blocks, C1-C3, and interconnecting buses between theblocks.

FIG. 21 is a logic diagram for block C1 of FIG. 20.

FIG. 22 shows the clock generation sequence for block C1 of thecontroller.

FIG. 23 is a block diagram for the controller module responsible forreceiving read data and comprising three blocks R1-R3, andinterconnecting buses between the blocks.

FIG. 24 is a logic diagram for a block R1 of a controller module forreceiving read data from a memory and inserting a programmable delay.

FIG. 25 is logic diagram for a block R2 of a controller module forcreating a clock for receiving read data.

FIG. 26 is logic diagram for a block R3 of a controller module forgenerating the value of a clock phase for receiving read data.

FIG. 27 shows a sequence of receive timing signals illustrating fourcases of alignment of a clock signal within a time interval and thegeneration of bus signals.

FIG. 28 shows a sequence of timing signals that illustrate how timingvalues are related and maintained in RXA and RXB registers.

FIG. 29 shows a sequence of receive timing signals that illustrate acalibration sequence for transferring bus signals.

FIG. 30 is a block diagram for a controller block T0, part of block C3,and responsible for transmitting write data.

FIG. 31 is a logic diagram for block T1, part of block T0, fortransmitting write data and inserting a programmable delay.

FIG. 32 is a logic diagram for block T2, part of block T0, for creatinga clock signal for transmitting write data.

FIG. 33 is a logic diagram for block T3, part of block T0, forgenerating the value of a clock phase for transmitting write data.

FIG. 34 shows a sequence of transmit timing signals illustrating fourcases of alignment of a clock signal within a time interval and thegeneration of bus signals.

FIG. 35 shows a sequence of timing signals that illustrate how timingvalues are related and maintained in TXA and TXB registers.

FIG. 36 shows a sequence of transmit timing signals that illustrate acalibration sequence for transferring bus signals.

FIG. 37 is a block diagram of the logic needed to perform thecalibration processes of a preferred embodiment of the presentinvention.

FIG. 38 is a block diagram for a memory system to implement a powerreduction mechanism.

FIG. 39 shows a sequence of timing signals for a read transaction in thememory system of FIG. 38.

FIG. 40 is another embodiment of a logic diagram of a controller modulefor createing a clock and control signals for receiving read data.

FIG. 41 is another embodiment of a logic diagram of a controller modulefor receiving read data from a memory and inserting a programmabledelay.

FIG. 42 is another embodiment of a logic diagram of a controller modulefor creating a clock signal and control signals for transmitting writedata.

FIG. 43 is another embodiment of a logic diagram of a controller modulefor transmitting write data and inserting a programmable delay.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The term “mesochronous” refers to a relationship between two signalshaving the same average rate or frequency, but which may havearbitrarily different phases. The term “mesochronous system” refers to aset of clocked components in which the clock signal for each clockedcomponent has the same frequency, but can have a relative phase offset.The term “static mesochronous system” means that the relative phaseoffsets are fixed and do not vary during normal system operation. Theapproach of using fixed relative phase offsets that do not vary duringnormal system operation has been the method practiced in prior artsystems. The present invention is directed to “dynamic mesochronoussystems” in which the phase offsets of the clocked components areallowed to drift over some range during system operation. The term“normal system operation” is used in this document to refer to ordinarymemory access operations, such as read, write and refresh operations.Calibration operations, which are used to determine timing offsetsrequired for successful command and data transmission between the memorycontroller and memory components of a memory systems, are generally notconsidered to be normal memory operations. In the preferred embodiment,the calibration hardware in the memory controller is configured toperform calibration operations periodically, and/or during periods oflittle memory system usage, and such calibration operations aregenerally separated by periods during which normal memory operations areperformed.

This document describes apparatus and techniques for managing the timingof signals within a system that can be beneficially applied to systemshaving a wide variety of signals, signal rates, time intervals, buses,signal-to-wire mappings, and so on. While the disclosed description willoften identify a preferred embodiment or implementation to illustrate aconcept, it should be understood that the concept is not limited to theembodiment or implementation employed in the discussion.

All of the variations of signal, symbol, bit, values, time intervals,buses, and signal-to-physical wire mapping are independent of themethods described in this document. This document describes a set oftechniques for managing the timing of signals within a system that canbe beneficially applied to all the variations described above. Thefollowing description will often choose a preferred variation toillustrate each concept. However, it should be understood that theconcept under discussion is not limited to the particular variation thatis employed in the discussion.

FIG. 2 shows an example of a prior art static mesochronous memory system200. The memory system comprises a controller 205 communicating to andfrom memory 210 via unidirectional bus 215 and bi-directional bus 216.Bus 215 carries address and control information from the controller tothe memory. Bus 216 carries data information from the controller to thememory during write operations and carries data information from thememory to the controller during read operations. Bus 215 is alsoreferred to here as the RQ bus, and bus 216 is also referred to here asthe DQ bus. This description will also refer to separate D and Q signalsets, i.e., signal sets 217, 218 at the controller and signal sets 219,220 at the memory, even though such signal sets share the same physicalwires in this system.

Memory component 210 in FIG. 2, is one component of a two dimensionalarray of memory components, with ranks (rows) indexed by the variable“i” and slices (columns) indexed by the variable “j”. Index values forthe ranks and slices are zero at the memory controller and assumeinteger values greater than zero as you move further away from thecontroller 205. In system 200, there is a single memory component ineach rank. Other memory systems may have more than one memory componentper rank. In this document, notation “[i,j]” is used to label a wire orbus at different physical points along a physical length. For example,the CTC[0,j] and CTC[i,j] signals are associated with two differentpoints along the same physical wire. The notation is used throughoutthis document so that a wire or bus can be labeled or identified at anypoint along a signal transmission path.

Each slice of the memory components is attached to the DQ and RQ busses,which carry data signals and request signals, and to bus 230 (forreceiving timing information or signals). Bus 230 (which is shown asthree sub-buses 230(a), 230(b) and 230(c)), communicating a clock signal(CLK), carries timing information from the controller 205 to the memorycomponent 210 so that information transfers on the other two buses canbe coordinated.

The controller uses internal clock signal CLKC for its internaloperations, including transmitting and receiving on the RQ and DQ buses.CLKC is routed so that it passes the memory controller and memorycomponents a total of three times. Clock signal 225 is made up of threesub-signals: CTE (clock-to-end), CTC (clock-to-controller), and CFC(clock-from-controller). The controller drives the CTE signal, whichtravels to the end of the slice of components, and returns as CTC. CTCenters the controller, and leaves the controller (unbuffered) as CFC,and then travels back to the end of the slice. While a single pair ofphysical wires generally carries the three clock signals (with CTC andCFC being carried by the same physical wire), three different signallines (i.e., wires) are shown in FIG. 2 for clarity.

CLKC is also used as a reference for the phase-loop lock (PLL) ordelay-loop lock (DLL) circuit 255 that drives CTE[0,j] signals onto bus230(a). The PLL/DLL circuit 255 internally generates the CTE[0,j] signaland adjusts the phase of the CTE[0,j] signal until the CTC[0,j] andCFC[0,j] signals have the same phase as the CLKC signal.

Memory components receive the CTC and CFC signals on buses 235 and 240,respectively. More specifically, each memory component has its own pairof clock input connections for receiving the CTC and CFC signals,respectively. The CTC[i,j] and CFC[i,j] signals that are received by amemory component [i,j] on buses 235, 240, respectively, have phases thatare offset from CLKC and from each other due to propagation delays. Thepropagation delays on the clock bus 230 are essentially identical topropagation delays on the RQ and DQ busses, and thus the CTC[i,j] andCFC[i,j] signals are used by the memory component [i,j] to control thetransmission of data signals on the DQ bus 216 and to control thereceipt of control and data signals from the RQ and DQ buses,respectively.

The CTC and CFC signals on buses 235, 240 pass through PLL/DLL circuits265, 270, respectively, within the memory component 210. Memorycomponents use PLL or DLL circuits to ensure that the clock domainswithin the memory are phase-aligned with the external memory clocksignals that provide the timing references. By phase aligning the clocksignals in this fashion, memory sub-components 245, 250 receive timinginformation for transmitting and receiving on internal D and Q buses219, 220.

Clock domain crossing logic 275 links the portions of the memorycomponent 210 that operate in two clock domains. One domain is used fortransmitting read data. The other domain is used for receiving writedata and address/control information. Each memory component needs topass information between the two clock domains, and this is controlledby the clock domain crossing logic 275.

FIG. 3 shows a timing diagram for system 200 (FIG. 2). The CLK signal225 drives the PLL/DLL 255, which produces CTE[0,j] after a delay oft_(PLL/DLL,). CTE[0,j] propagates to the end of the wire 230(a) andreturns as the CTC signal to memory component [i,j]. The CTC[i,j] signalis delayed by t_(PROPtoEND) relative to the CTE[0,j] signal. Read dataQ[i,j] transmitted by the memory component to the controller issynchronized by the memory component [i,j] with the CTC[i,j] clock. Theread data Q[i,j] signal set and the CTC[i,j] clock signal requires anadditional delay of t_(PROPij) to reach the controller to become Q[0,j]and CTC[0,j].

The PLL/DLL 255 adjusts the t_(PLL/DLL) delay so that the CTC[0,j] andCLK signals have the same phase alignment. This means thatt_(PLL/DLL)+t_(PROPtoEND)+t_(PROPij)=C*t_(CYCLE), where C is an integerand denotes the number of CLK cycles required for the round trip of theclock signal from CTE[0,j] to CTC[0,j]. The CTC[0,j] signal becomes theCFC[0,j] signal and exits the controller. After a delay of t_(PROPij)CFC[0,j] reaches a memory component as CFC[i,j], where it is used toreceive the RQ[i,j] address/control information and D[i,j] write datainformation. The information on these two buses are delayed byt_(PROPij) from the RQ[0,j] and D[0,j] buses, respectively.

In system 200, the controller is able to perform all transfer operationswithin the single clock domain of the CLKC signal. Each memorycomponent, on the other hand, operates in two clock domains, as notedabove. One domain is earlier in time than CLKC by t_(PROPij) and is usedfor transmitting read data. The other domain is later than CLKC byt_(PROPij) and is used for receiving write data and address/controlinformation.

Note that in all of these cases of transfers on the RQ and DQ buses, thesampling (e.g., rising) edge of the clock that is used to receive a setof bits from a bus is shown as being aligned with the start of the validwindow of the bits. In practice, the receiving clock will have itssampling edge aligned with the center of the valid window of the bits.The clock recovery circuits (PLL or DLL) present on both the controllerand memory component can perform this alignment easily. For simplicity,this detail is not shown in the timing diagram. Although other staticalignments of the clock with respect to the data may be used, one keypoint with respect to static systems is that the alignment does notchange during system operation.

Phase offsets required for normal operation of system 200 are known orcan be generated automatically by the system hardware. Thisdetermination of phase offsets can be done because the clock signalstravel through a path that is essentially the same as the path of thedata and address/control signals for which they provide a timingreference. The determination of phase offsets also requires that PLL orDLL circuits be used to maintain the static phase relationships.

In practice, such PLL or DLL circuits will not align the phase of twosignals exactly; there will be some small error due to, for example,circuit jitter. Such jitter must be factored into the overall timingbudget for transferring each bit of information on a signal. This timingbudget for transmitting and receiving a bit includes, for instance,setup and hold times of receive circuits and the variation of the outputvalid delay of the transmit circuits. However, the timing budget doesnot include a component for the round-trip propagation delay2*t_(PROPij) between the memory component and memory controller. Theround-trip propagation delay 2*t_(PROPij) is accounted for in the clockdomain crossing logic 275. The accounting for round-trip propagationdelay 2*t_(PROPij) increases the latency of a read operation, but doesnot impact the bandwidth of read and write transfers. Such transferswill occur at a bandwidth that is determined by the circuits of thecomponents, and not by the length of the wires that connect thecomponents. The bandwidth determination based on circuit components is acritical factor in the advantage of static mesochronous systems oversynchronous systems in which all components use clock signals that areat essentially the same phase. Since the transfers in the synchronoussystem must include the propagation delay term in the timing budget fora bit transfer, such inclusion of the delay term limits the bandwidth oftransfers.

FIG. 4 shows a second example of a prior art static mesochronous memorysystem 300 comprising a controller 305 communicating to and from amemory component 310, which may be in an array of similar memorycomponents. Like system 200, memory components form a two dimensionalarray, with ranks (rows) indexed by the variable “i” and slices(columns) indexed by the variable “j”. As throughout this document, theindex values are zero at the memory controller, and assume integervalues greater than zero as you move further away from the controller.For example, in a system with a 2×3 array of memory components (i.e.,three memory components in each of two ranks) the value of the “i”variable increases from zero at the controller to “1” for each of thethree memory components in the first rank and to “2” for each of thethree memory components in the second rank. Furthermore, in thisexemplary system the value “j” increases from zero at the controller to“1” for each of the two memory components in the first slice, to “2” foreach of the two memory components in the second slice, and to “3” foreach of the two memory components in the third slice.

Unlike system 200, in system 300 there can be more than a single memorycomponent in each rank. Each rank of memory components is attached to anRQ bus 315 and a DQ bus 320. Other ranks are connected to different RQbuses and other slices are connected to different DQ buses. The RQ bus315 is unidirectional and carries address and control information fromthe controller to the rank of memory components. The CLK bus 317 isunidirectional and carries timing information from the controller to thememory components so that information transfers on the other two busescan be coordinated. The CLK bus 317 has the same topology as the RQ busit accompanies. The slice of memory components is attached to DQ bus320, which connects the controller and memory and is bi-directional, andcarries data information from the controller to a memory componentduring write operations, and carries data information from a memorycomponent to the memory controller during read operation.

The controller uses an internal clock CLKC, carried by internal bus 325,for its internal operations, including transmitting on the RQ bus 315.The CLKC signal is also used as a reference for the PLL or DLL circuit330 that drives CLK[i,0]. The PLL/DLL 330 adjusts the phase of theCLK[i,0] signal to have the same phase as CLKC.

At the memory component 310, the CLK[i,j] signal is received offset inphase by a PLL or DLL circuit 335, which produces a buffered internalclock that is of essentially the same phase as received. This internalbuffered clock signal is used to control transmission of read data ontothe DQ bus 320 and to control the receipt of control and data signalsfrom the RQ and DQ buses, as appropriate, generally to control thetiming of operations performed by internal memory sub-components 340,345 and 350. Because there is a single clock domain inside the memorycomponent 310, there is no need for clock domain crossing logic in thememory component 310 as there was in system 200.

Instead, the clock domain crossing logic has been shifted to thecontroller 305 as clock domain crossing logic 375. The shift in logic375 to the controller is because the write data D[0,j] transmit logic365 and the read data Q[0,j] receive logic 370, must be operated in twoclock domains, CLKD[0,j] and CLKQ[0,j], that have different phases thanthe CLKC domain used by the rest of the controller. As a result, phaseadjustment logic is needed that delays CLKC by t_(Dij) and t_(Qij) toform the CLKD[0,j] and CLKQ[0,j] signals, respectively. The phaseadjustment logic for the CLKD[0,j] and CLKQ[0,j] signals, are shown ascircuit elements 380 and 355, respectively.

FIG. 5 shows a timing diagram for system 300 (FIG. 4). The CLK signaldrives the PLL/DLL 330 to produce CLK[i,0]. The CLK[i,0] signal isdelayed by t_(PROPCLKij) as it propagates to memory component [i,j] tobecome signal CLK[i,j]. The RQ[i,0] signal transmitted by controllerelement 360 is delayed by t_(PROPRQij) as it propagates to memorycomponent [i,j] to become the RQ[i,j] signal at internal memorycomponent 350. The CLK and RQ buses are routed together, so that the twopropagation delays are essentially the same.

In the controller, phase adjustment logic 380 delays CLK by t_(Dij) toform the CLKD[0,j] signal. This clock signal is used by the controller'sdata transmit element 365 to control the phase of the write data D[0,j].The D[0,j] signal set is delayed by t_(PROPDij) as it propagates tomemory component [i,j] to become signal set D[i,j]. The controllerselects t_(Dij) so that t_(PROPCLKij)=t_(Dij)+t_(PROPDij). There is alsophase adjustment logic 355 in the controller that delays CLK by t_(Qij)to form the CLKQ[0,j] signal. This clock signal is used to receive theread data Q[0,j] at 370. The Q[i,j] signal set is delayed by t_(PROPQij)as it propagates from memory component [i,j] to become signal setQ[0,j]. The controller selects t_(Qij) so thatt_(Qij)=t_(PROPCLKij)+t_(PROPQij).

In the transfers on the RQ and DQ buses, the sampling (rising) edge ofthe clock that is used to receive a set of bits (or more generally,symbols) on a bus is shown aligned with the start of the valid window ofthe bits. In the real system, the receiving clock will typically haveits sampling edge aligned with the center of the valid window of thebits. The clock recovery circuitry (PLL or DLL) that is present on boththe memory controller and memory component can perform this alignmenteasily. Other static phase alignments of the signals on the CLK, RQ, andDQ buses are also possible. Memory components are able to perform alltransfer operations within the single clock domain of the internalbuffered clock signal. Each memory component will operate in its ownunique clock domain. The phase of each clock domain will stay fixedrelative to the phase of CLK in the controller; hence the reason for theterm “static” mesochronous system.

The controller, on the other hand, has multiple clock domains. CLK isthe principle domain, and CLKD[0,j] and CLKQ[0,j] are the domains (twofor each slice [j]) used for transmitting and receiving, respectively.The phase offsets of CLKD[0,j] and CLKQ[0,j] are dependent, for example,upon the lengths of the wires that connect the components, and may havea range that is greater than the cycle time of CLK. Typically, the rangeof phase offsets (t_(Dij) and t_(Qij)) for CLKD and CLKQ is many timesgreater than the cycle time of CLK. The domain crossing logic 375 mustaccommodate these ranges of phase offsets.

Typically, the phase offsets or adjustment values t_(Dij) and t_(Qij)are determined at system initialization time. The values are stored andthen used during normal system operation. Each rank [i] in the systemneeds its own set of phase adjustment values t_(Dij) and t_(Qij) thatmust be loaded prior to transferring data to or from a memory device inthe rank. In practice, the PLL or DLL circuits of system 300 do notalign the phase of two signals exactly, as there is always at least asmall error because of circuit jitter, as described earlier. This jittermust be absorbed into the overall timing budget for transferring eachbit of information.

As with system 200, the timing budget for transmitting and receiving abit for system 300 does not include a budget allocation for theround-trip propagation delay, 2*t_(PROPij), between the memory componentand controller. Instead, the round-trip propagation delay, 2*t_(PROPij),is accounted for by the clock domain crossing logic 375. The round-trippropagation delay increases the latency of read operations, but does notimpact the bandwidth of read and write transfers. The transfers occur ata bandwidth that is determined by the circuits of the components, andnot by the length of the signal wires that connect the components.

Dynamic Mesochronous Memory System

FIG. 6 shows an overview of a preferred embodiment of a dynamicmesochronous memory system 400 in accordance with the present invention.This system is topologically similar to system 300 with respect to theinterconnection of components by buses and comprises a controller 405for communicating to and from the memory components 410. In this system,there can be more than a single memory component in each rank and/or ineach slice. Unless otherwise noted or described, reference numbers thatdiffer by 100 for systems 300 and 400 represent circuit elements at thesame topological locations and having at least some functional aspectsin common, even if their internal design and operation differssubstantially.

While the preferred embodiments will be described in terms of a memorycontroller and memory components, it is to be understood that the term“memory controller” includes any device that performs the functions of amemory controller described herein, and that the term “memory component”includes any device that performs the functions of a memory component ordevice described herein. For instance, if the functions of a memorycontroller are integrated with a central processing unit or with anothercontroller device, the resulting device will be considered a “memorycontroller” in the context of the present invention. Similarly, if thememory storage, access and calibration functions of a memory device areintegrated into another device, such as an application specificintegrated circuit (ASIC), that device will be considered a “memorycomponent” in the context of the present invention.

System 400 differs from system 300 in numerous respects. The followingis a partial listing of significant differences between the two systems:(1) the memory component 410 has a clock buffer 443 instead of thePLL/DLL clock recovery circuits 330 and 335 of memory component 310; theclock buffer can have varying delay during system operation; (2) unlikememory component 310 and controller 305, the memory component 410 andcontroller 405 include calibration logic 485 and 490, respectively, tosupport a calibration process; (3) controller 405 includes enhancedclock domain crossing logic 475 to improve the efficiency of thecalibration process. In addition, certain elements of the memorycomponent 410, such as the RQ and DQ handling logic 450, 445, 440include new logic or circuitry to support the calibration process. Thesenew aspects of the memory component 410 and controller 405, as well asmany others, are discussed below.

The clock signal used to time the transmission of signals (sometimescalled requests) on the RQ[i,0] bus 415 is called CLK[i,0]. CLK[i,0] hasessentially the same phase as CLKC. The CLK[i,j] signal that is receivedby memory component [i,j] will have a phase that is offset from CLKC.This CLK[i,j] signal is received by a simple clock buffer 443, which ismuch less complex and consumes much less power than the PLL or DLLcircuits 330 and 335 in system 300. Buffer 443 produces a bufferedinternal clock CLKB[i,j] at 444 that is at a different phase than theCLK[i,j] signal, received by buffer 443. The CLKB[i,j] signal is used totransmit data onto the DQ bus 420, to receive signals (e.g., requestsand data) from the RQ and DQ buses 415, 420, respectively, and toperform all other internal operations in the memory 410. Because thereis a single clock domain inside the memory, there is no need for clockdomain crossing logic in the memory component, such as logic 275 ofsystem 200.

Additionally, system 400 includes calibration logic 485 and 490.Calibration logic 485 is added to the memory 410, and logic 490 is addedto the controller 405. The calibration logic circuits 485 and 490operate in conjunction with one another so that delay variations duringsystem operation are detected and complementary delay elements in logic480, 455 are adjusted.

As in system 300, the write data D[0,j] transmit logic 465 and the readdata Q[0,j] receive logic 470 must be operated in two clock domains,CLKD[0,j] and CLKQ[0,j], having different phases than the CLK signaldomain used by the rest of the controller. As a result, there is phaseadjustment logic 480 and 455 that delays CLK by t_(Dij) and t_(Qij),respectively, to form the CLKD[0,j] and CLKQ[0,j] signals.

Also as in system 300, the values t_(Dij) and t_(Qij) are functions ofthe propagation delay parameters t_(PROPCLKij), t_(PROPDij), andt_(PROPQij). These propagation delay parameters are relativelyinsensitive to temperature and supply voltage changes. In system 300,once the values t_(Dij) and t_(Qij) have been generated during systeminitialization, they may be left static (unchanged) during systemoperation. But in system 400, the values t_(Dij) and t_(Qij) are also afunction of the delay of the clock buffer t_(Bij) (as well as delay ofother circuits such as the transmitters and receivers). This clockbuffer delay will change during system operation because it isrelatively sensitive to temperature and supply voltage changes. Thevalues t_(Dij) and t_(Qij) that are generated during systeminitialization are dynamic (changing) during system operation, and acalibration process, using the calibration logic 485 and 490, keeps thevalues t_(Dij) and t_(Qij) updated.

In addition, enhancements are made to the clock domain crossing logic475 in system 400 (relative to system 300) to improve the efficiency ofthe calibration process and so that the calibration process can behandled completely by hardware. Because the enhancement (i.e., hardwarefor implementing the calibration process) is implemented in hardware,primarily in the controller but also in the memory components, theperformance of the system 400 is not significantly impacted by theoverhead of the calibration process.

Within the memory component 410, receiver and transmitter circuits 440,445, and 450 are able to perform all transfer operations within a singleclock domain that is defined by the internal buffered clock signalgenerated by buffer 443. A clock domain is defined by a set of one ormore clock signals that have the same frequency and phase. Forconvenience, a clock domain is often named using the name of the clocksignal that defines the clock domain (e.g., “the CLK clock domain” is aclock domain defined by the CLK clock signal). Each memory componentoperates in its own clock domain, which may be unique for each memorycomponent. The phase of the clock domain for each memory component canchange relative to the phase of CLK in the controller; hence, the term“dynamic” mesochronous memory system.

The controller 405, on the other hand, has multiple clock domainsdefined by the CLK, CLKD[0,j] and CLKQ[0,j] clock signals. CLK on bus425 is considered a principle clock domain, and CLKD[0,j] and CLKQ[0,j]are derived clock domains in that they are based on CLK by way of phaseadjustment logic 480 and 455, respectively. CLKD[0,j] and CLKQ[0,j] areused for transmitting and receiving, respectively. The phase offsetst_(Dij) and t_(Qij) of CLKD[0,j] and CLKQ[0,j] are dependent upon thelengths of the wires that connect the components, parasitic capacitancealong these wires, and upon the changing delay t_(Bij) of the memorycomponent clock buffer 443. These phase offsets may have a range that isgreater than the cycle time of CLK. In some cases the range may be manytimes greater. The domain crossing logic 475 accommodates these rangesof phase offsets and handles the phase offset range in hardware duringcalibration process updates.

Each rank [i] in the system will have its own set of phase adjustmentvalues t_(Dij) and t_(Qij) that are loaded prior to transferring data toor from the rank. Each set of values are kept updated by the calibrationprocess involving logic 485, 490.

The dynamic mesochronous system 400 has similar timing benefits to thatof the static mesochronous system 300. For example, in system 400 thetiming budget for transmitting and receiving a bit does not include thepropagation delay t_(PROPij) between the memory component andcontroller. Instead, the round-trip propagation delay 2*t_(PROPij) isaccounted for in the clock domain crossing logic 475. This round-trippropagation delay increases the latency of a read operation, but doesnot impact the bandwidth of read and write transfers. The transfersoccur at a bandwidth that is determined by the circuits of the memorycontroller and memory components, and not by the length of the signalwires that connect the memory controller and memory components.

As noted, the clock recovery circuit 335 of the memory 310 is replacedby a simple clock buffer 443 in system 400. This change in system 400results in a number of benefits. First, circuit area on the memorycomponent is reduced. Additionally, the design complexity of the memorycomponent is substantially reduced, particularly as the clock recoverycircuit 335 often is a complex part of the memory design. Further, thestandby power of the clock recovery circuit is eliminated. Standby powerrefers to the power dissipated when there are no read or write transferstaking place. Typically a PLL or DLL must dissipate some minimum amountof power to keep the output clock in phase with the input clock. Inpractice, this standby power requirement has made memory components witha DLL or PLL difficult to use in portable applications, where standbypower is important.

System 400 introduces a memory system topology based on dynamicmesochronous clocking. A number of variations in system topology,element composition, and memory component organization are possible, andsome preferred and representative variations will be described.Individual variations may, in general, be combined with any of theothers to form composite variations. Any of the alternate systems formedfrom the composite variations can benefit from the method of dynamicmesochronous clocking.

FIG. 7 shows a baseline memory system topology 700. Topology 700 issimilar to the topology of system 400, but with some modifications asdescribed below.

The memory controller 705 is shown in FIG. 7. A single memory port 710,(labeled Port[1]) is shown, but the controller 705 could have additionalmemory ports. In general, a controller has other external signals andbuses that are not directly related to the memory system(s), which forclarity purposes are not shown in FIG. 7.

The port 710 of the controller consists of two types of buses: the X busand the Y bus. The X and Y buses are composed of wires for carryingdifferent sets of signals and have different routing paths through thememory system. The X bus is depicted as comprising NS X buses shown as715, 716, 717 and 718. System 700 also depicts the Y bus comprising theNM Y buses, shown as buses 720, 721. The NS X buses usually carries datasignals and the NM Y buses usually carries address signals, but othersignal information configurations are possible. NM and NS are integershaving values greater than zero.

Each of the NS X buses connect to the memory components along one“slice” (column). For example, the memory components along a slice areshown as components 730, 732, 734 and 736 in memory module 740. Asshown, each of the NS X buses connect to one of the NS slices of each ofthe NM memory modules 740, 750. Typically, only the memory componentsalong one slice will be active at a time, although variations to thisare possible.

There are NM of the Y buses, with each Y bus connecting memorycomponents on one “module” (set of ranks). For example, the memorycomponents of the first rank (e.g., the leftmost rank shown in FIG. 7)are shown as components 730, 744, 746 and 748. Each of the NM memorymodules can consist of NR ranks (rows) of memory components. Typically,all of the memory components of one rank of one module will be active ata time, although variations to this are possible. NM and NR are integershaving values greater than zero. In some systems, the memory system mayconsist of NR ranks of memory components attached to the same printedcircuit board (also called a wiring board) that holds the memorycontroller 705.

Each of the NM modules 740, 750 have a dedicated Y bus, i.e., one of theNM Y buses 720, 721, but typically most or all of the signals on a Y busare duplicates of the signals on other Y buses. Some signals carried onthe NM Y buses may be used to perform module or rank selection. Theseselection signals are generally not duplicated, but are dedicated to aparticular module or rank.

Likewise, each of the NR ranks on a module connects to the module'sdedicated Y bus. Typically, most or all of the signals composing the Ybus are connected to the memory components of each rank. Some signals onthe NR ranks are used to perform rank selection. The rank selectionssignals are generally not duplicated, and connect to only the memorycomponent(s) of one rank.

Generally, all signals transmitted on the X and Y buses are operated atthe maximum signaling rates permitted by the signaling technologyutilized. Maximum signaling rates often rely on the sequentialconnection of memory components to a physical wire by short stub wiresbranching from the physical wire. Maximum signaling rates also implycareful impedance matching when signals are split (a physical wirebecomes two physical wires) and when signals are terminated (the end ofa physical wire is reached).

The Y bus signals on a module may pass through splitting elements(labeled “S” in the figures) in order to make duplicate copies ofsignals. Alternatively, the signals may connect sequentially to thememory components of one or more of the NR ranks (the figure showssequential connection to two ranks). A module 740, 750 may contain asfew as one or two ranks, in which case no splitting element is needed onthe module at all.

Sample splitting element variations are shown in FIGS. 10(a)-(d).

Returning to FIG. 7, the Y bus signals connect to termination elements760 (labeled “T” in the figures) where the signals reach the end of arank. Y bus signals are typically unidirectional, so terminationelements are shown only at the memory end of the signals. If any Y bussignals were bi-directional, or if any Y bus signals drove informationfrom memory to controller, then termination elements would be requiredon the controller end of the Y buses.

Sample bus termination element variations 760 are shown in FIGS.1(a)-(d).

Returning to FIG. 7, the X bus signals can pass through a splittingelement on the same printed circuit board that holds the controller 705.One of the duplicate signals from a splitter, such as from splitter 752,754, 756 or 758, enters one of the NM modules, such as module 740, andthe other continues on the printed circuit board to the next module. TheX bus signals connect sequentially to the memory components of eachslice and end at a termination element such as on bus 765.Alternatively, if the system only contains a single memory module, nosplitting elements would be needed for the X bus signals.

The X bus signals are typically bi-directional, so termination elementsare needed at the controller end of each signal (e.g., where atermination element 762 connects to bus 718) as well as at the far endof the memory array. For any unidirectional X bus, termination elementswould be required only at the end of the X bus that is opposite from thecomponent that drives the signal.

Typically, all of the signals on the X bus are transmitted (or received)by all the memory components of each slice. In some embodiments, theremay be some signals on the X bus that are transmitted (or received) byonly a subset of the memory components of a slice.

FIG. 8A shows a variation on the Y bus topology in which the controller805 drives a single Y bus 810 to all the modules. A splitting element812 is used to tap off a duplicate Y bus signal bus for each memorymodule, such as modules 820, 830. X bus splitters 816 sequentiallyconnect the slices of each module. The use of external terminationelements 818 is desirable when any X bus signals (on X buses X₁ toX_(NS), not separately shown) are bi-directional. Internal (i.e.,internal to the modules 820, 822) splitter 822 and termination elements824 are still used for each slice of each of the NM modules having NRranks of memory components 828. In system 800 the controller drivesfewer buses, but each Y bus signal will pass through a larger number ofsplitting elements. This increase in the number of splitting elementsmay impact signal integrity or propagation delay.

FIG. 8B shows a second variation on the Y bus topology in which thecontroller 855 drives the Y buses on the same group of physical wires856, 858 as the X buses to the modules. In other words, the Y buses runparallel to the X buses in this embodiment. In system 850, there are nobuses flowing along each of the NR ranks. There may be some signals inthe X or Y bus used to perform a rank and module selection which connectto only a subset of the memory of a slice (only two slices are shown forsimplicity). Alternatively, module and rank selection may be performedby comparing X or Y bus signals to an internal storage circuit (whichmay be located in each memory component) that contains the module orrank identification information. This method of module and rankselection could be used in any of the other topology variations. As withFIG. 8A, external termination elements 860 and external splitterelements 862 may be used, but only internal termination elements 864 areneeded for each slice of the NM modules 870 having NR ranks of memorycomponents 875.

System 900 in FIG. 9 shows a variation on the X bus topology of system700 in which each X bus signal (e.g., on X bus 925) passes through oneset of pins on each module and exits through a different set of pins.Controller 905 transmits X bus signals to memory modules 910, 920, andNM Y buses 930, 940 connect to each of the NM modules. No externalsplitting element is needed on the main printed circuit board, and fewerinternal termination elements 935 are needed on the modules. While extrapins are needed on each module, there is a reduction in the number ofsplitting and termination elements.

FIGS. 10A-10D show some of the possible splitting element variations. Ineach of these figures, splitter element variations are shown where asingle signal is split into two signals. In FIG. 10A, a splitter 1000converts a single signal labeled “1” into two signals labeled “2”, bythe use of a clocked 1010 or unclocked 1020 buffer. In FIG. 10B, thesignals are bi-directional, and signals can be split or combined.Generally, a single signal can pass through an enabled switch or bufferof the splitter 1030 to form two signals and any port can receive asignal component from any other port. Splitter element 1030 is abi-directional buffer consisting of either a pass-through, non-restoringactive switch 1035 with an enable control, or a pair of restoringbuffers 1042, 1044 with a pair of enable controls in element 1040. Notethat element 1030 could also be used for unidirectional signals.

Splitter element 1050 (FIG. 10C) is a unidirectional resistive device,implemented from either active or passive components. Splitter 1055permits an attenuated signal to be passed to one of the output ports(labeled “2”), with the resistive value, R_(DAMP), chosen to limit theimpedance mismatching that occurs. An alternative method would allow thecharacteristic impedance of the traces to be varied so that (incombination with a series resistance on one output port) there would bea smaller mismatch for unidirectional signals. Element 1060 (FIG. 10D)is a bi-directional power-splitting device, allowing the impedance to beclosely matched for a signal originating on any of the three ports. Theresistive devices, Z₀ in element 1065 or Z₀/3 in element 1070, could beimplemented from passive or active devices. FIGS. 10C and 10D aresimilar to FIG. 10B in that a signal input at any port can yield signalsat the remaining two ports. Splitter element 1050 (FIG. 10C) utilizes awire stub with series damping, and splitter element 1060 (FIG. 10D)utilizes an impedance-matching splitter. Like splitter element 1030(FIG. 10B), the splitter elements 1050 and 1060 have bi-directionalports so that any port can be an input port and any port can receive asignal component from any other port.

FIGS. 11A to 11D show some of the possible termination elementvariations. Element 1100 (FIG. 11A) is a passive, external terminationelement. The element may be implemented, for example, as a single deviceconnected to a single termination voltage, V_(TERM), or as two (or more)devices R₁ and R₂ in element 1108 connected to two (or more) terminationvoltages, such as V_(DD) and circuit ground. The termination element1100 resides on a memory module or on a main printed circuit board.

Termination element 1120, shown in FIG. 11B, is an active, externaltermination element. It may be implemented, for example, as a singledevice 1110 in a termination element 1125 connected to a singletermination voltage V_(TERM), or as two (or more) devices 1120 and 11320in a termination element 115 connected to two (or more) terminationvoltages, such as V_(DD) and circuit ground. The termination element1120 resides on a memory module or on a main printed circuit board. Thevoltage-current relationship needed for proper termination is generatedby control voltage(s), which are maintained by an external circuit.Typically, the external circuit (not shown) measures a value thatindicates whether the voltage-current relationship is optimal. If it isnot, the external circuit makes an adjustment so the voltage-currentrelationship becomes more optimal.

Termination element 1160, shown in FIG. 11C, is a passive, internaltermination element. This variation is similar to termination element1100 except that termination element 1160 resides inside a memorycomponent or inside the memory controller, both shown as component 1165.Termination element 1170, shown in FIG. 11D, is an active, internaltermination element. The FIG. 11D variation is similar to terminationelement 1120 except that element 1170 resides inside a memory componentor inside the memory controller, both shown as component 1175.

FIG. 12 shows a baseline system configuration 1200 for the memorycomponent “M” that is shown in the system topology diagrams, such aselement 730 in system 700. An X bus 1205 and a Y bus 1210 connect tomemory component M. The X and Y buses correspond to the X and Y busesshown in topology FIGS. 7, 8A, 8B and 9. The memory component M containsinterface logic for receiving and transmitting the signals carried bythe X and Y buses. The memory component M also contains a memory core1215 that consists of 2^(Nb) independent banks. Here Nb is the number ofbank address bits and is an integer greater than or equal to zero. Thebanks are capable of performing operations independent of one another,as long as the operations do not have resource conflicts, such as thesimultaneous use of shared interface signals.

The Y Bus 1210 carries two signal sets: the row signal set 1220-1228 andthe column signal set 1230-1238. Each group contains a timing signal(A_(RCLK) 1220 and A_(CCLK) 1230), an enable signal (A_(REN) 1222 andA_(CEN) 1232), an operation code signal set (OP_(R) 1224 and OP_(C)1234), a bank address signal set (A_(BR) 1226 and A_(CR) 1236), and arow or column address signal set (A_(R) 1228 and A_(C) 1238). The numberof signals carried by the signal sets are represented with a “/P”, suchas Nopr/P, Nopc/P, Nb/P, Nb/P, Nr/P, and Nc/P, respectively. The factor“P” is a serialization or multiplexing factor, indicating how many bitsof a field are received serially on each signal. The demultiplexers 1240and 1245 convert serial bits into parallel form. The P factors for Nopr,Nopc, Nr, Nc, and P may be integer values greater than zero. Forexample, there might be eight column address bits transmitted as twosignals for the column address signal set, meaning that four columnaddress bits are received sequentially on each signal. The P factor forthis example would be four. Memory component 1200 (i.e., the baselinememory component) uses the same P factor for all the sub-buses of the Ybus, but different factor values could also be used for differentsub-buses in the same memory component. Here, P is an integer greaterthan zero.

It is also possible that the signal sets could be multiplexed onto thesame wires. The operation codes could be used to indicate which signalset is being received. For example, the bank address signal sets couldshare one set of wires, and the row and column address signal sets couldshare a second set of wires, and the operation code signal sets couldshare a third set of wires.

The six signal sets (i.e., signals 1224-28, 1234-38) are received bycircuitry in the memory component 1200 that uses the timing signals(A_(RCLK) and A_(CCLK)) as a timing reference for when a bit is presenton a signal. These timing signals, for example, could be a periodicclock or they could be a non-periodic strobe. An event (i.e., a risingor falling edge) could correspond to each bit, or each event couldsignify the presence of two or more sequential bits (with clock recoverycircuitry creating two or more timing events from one). In someimplementations the six signal sets share a single timing signal.

The enable signals 1222 and 1232 indicate when the memory system 1200needs to receive information on the associated signal sets. For example,an enable signal may be used to pass or block the timing signalsentering the memory component, depending on the value of the enablesignal, or an enable signal might cause the operation code signal set tobe interpreted as no-operation, or the enable signal may be used bylogic circuitry to prevent information from being received when thevalue of the enable signal indicates that such information is not forreceipt of the memory component 1200.

The enable signals can be used to select a first group of memorycomponents and to deselect a second group, so that an operation will beperformed by only the first group. For example, the enable signals canbe used for rank selection or deselection. The enable signals can alsobe used to manage the power dissipation of a memory component in system1200 by managing power state transitions. In some embodiments, theenable signals for the row and column signal groups could be shared.Further, each enable signal shown could be decoded or formed from two ormore signals to facilitate the task of component selection and powermanagement.

The de-multiplexed row operation code, row bank address, and row addressare decoded by decoders 1250, 1252 and 1254, and one of the 2^(Nb)independent banks is selected for a row operation. A row operation mayinclude sense or precharge operations. In a sense operation, one of the2^(Nr) rows contained in a bank selected by outputs of the decoders iscoupled to a column sense amplifier for the bank. For a prechargeoperation, a selected bank and its column sense amplifier are returnedto a precharged state, ready for another sense operation.

The de-multiplexed column operation code, column bank address, andcolumn address are decoded via decoders 1256, 1258 and 1260, and one ofthe 2^(Nb) independent banks is selected for a column operation such asread or write. A column operation may only be performed upon a bank thathas been sensed (not precharged). For a read operation, one of the2^(Nc) columns (with Ndq bits) contained in a column sense amplifierportion of the selected bank is read and transmitted on the Q signal set1290/1268. For a write operation, Ndq bits received on the D signal set(i.e., signals 1270-76) is written into one of the 2^(Nc) columnscontained in the column sense amplifier portion of the selected bank,using the Nm mask bits on sub-bus 1292 to control which bits are writtenand which are left unchanged.

The X bus 1205 carries two sets of signals: the read signal set and thewrite signal set. The read signals include signals 1262-1268, and thewrite signals include signals 1270-1276. Each group contains a timingsignal (Q_(CLK) 1264 and D_(CLK) 1274), an enable signal (Q_(EN) 1262and D_(EN) 1276), a mark or mask signal set (Q_(M) 1266 and D_(M) 1270,respectively), and a data signal set (Q 1268 and D 1272). The number ofsignals in the signal sets are represented with a “/N”, such as Ndq/N,Nm/N, and Ndq/N, respectively. The factor “N” is a serialization ormultiplexing factor, indicating how many bits of a field are received ortransmitted serially on each signal. The “mux” and “demux” blocksconverts the bits from parallel-to-serial and serial-to-parallel form,respectively. The parameters Ndqr, Nm, and N may contain integer valuesgreater than zero. This baseline memory system 1200 assumes that theread and write data signal sets have the same number of signals and usethe same multiplexing factors. This might not be true in other memorycomponents, and therefore the number of signals in each signal set mayvary. In some embodiment, the read and write data signal sets aremultiplexed onto the same wires.

The mark signal set provides a timing mark through mark logic 1286 toindicate the presence of read data. The mark signal set might have thesame timing as the read data signal set, or it might be different. Themask signal set 1292 indicates whether a group of write data signalsshould be written or not written as determined by mask logic 1288. Thisbaseline memory system assumes that the mark and mask signal sets havethe same number of signals and use the same multiplexing factors. Thisassumption might not hold true in other embodiments. It is also possiblethat, in other embodiments, the mark and mask data signal sets could bemultiplexed onto the same wires. In other embodiments, one or both ofthe mark and mask signal sets might not be implemented.

The data signal sets 1290, 1294 are received by circuitry 1284, 1278that uses the timing signals (Q_(CLK) and D_(CLK)) as a timing referencefor when a bit is present on a signal. These timing signals could be,for example, a periodic clock, or they could be a non-periodic strobe orany other timing reference. An event (e.g., a rising or falling edge ofa timing signal) could correspond to each bit, or each event couldsignify the presence of two or more sequential bits (with clock recoverycircuitry creating two or more timing events from each received event).It is possible that the data signal sets could share a single timingsignal. It is also possible that the X and Y buses could share a singletiming signal.

The enable signals Q_(EN) and D_(EN) 1262 and 1276 indicate when thememory component needs to receive information on the associated signalsets. For example, an enable signal might pass or block the timingsignals entering the memory component, or it may be used to preventinformation from being transmitted or received. The enable signals canbe used for slice selection or for managing power state transitions.

FIG. 13 shows the overview topology of an alternate preferred dynamicmesochronous memory system 1300. System 1300 is topologically similar tosystem 400 with respect to the interconnection of components by buses.For example, memory components would still form a two dimensional array,with ranks (rows) indexed by the variable “i” and slices (columns)indexed by the variable “j”, following the same notation as before. Insystem 1300, like system 400, there can be more than a single memorycomponent in each rank. Each rank of memory components is attached to anRQ bus 1315 and a CLK bus 1320. RQ bus 1315 is unidirectional andcarries address and control information from the controller to thememory components. CLK bus 1320 is unidirectional and carries timinginformation from the controller 1305 to the memory components 1310 sothat information transfers on the other two signals sets can becoordinated.

Each slice of memory is attached to a DQ bus 1325. DQ bus 1325 isbi-directional, and carries data information from the controller 1305 toa memory component 1310 during write operations, and carries datainformation from a memory component to the controller during readoperation. This description will also refer to the D and Q signal setsseparately to include signal sets 1327 and 1329, respectively, eventhough the same physical wires are shared.

The controller uses an internal clock signal CLK1 1330 for its internaloperations. The CLK1 signal is also used as a reference to drive theCLK[i,0] signal on bus 1320, CLKD[0,j] signal on bus 1332 and CLKQ[0,j]signal on bus 1334. The frequency of the clock signals such as CLK4 andCLK[i,0] is an integer multiple of the frequency of CLK1 by way offrequency multiplier 1335 (which is a 4×frequency multiplier in apreferred embodiment). This multiplication is done so that the frequencyof CLK[i,0] matches the frequency of the clock used to transmit andreceive write data D and read data Q.

The clock signal CLK1 is used to transmit signals on the RQ[i,0] bus1315. When frequency multiplier 1335 is a 4× multiplier, the rate atwhich bits are transferred on the RQ bus is one fourth the rate at whichbits are transferred on the D and Q signal sets. This transfer ratedifferential is consistent with the fact that a relatively small amountof address and control information is needed for transferring arelatively large block of read or write data. Other transfer ratedifferentials between RQ and DQ are possible.

The CLK[i,j] signal 1340 that is received by memory component [i,j] 1310will have a phase that is offset from CLK[i,0]. The CLK[i,j] signal isreceived by a simple clock buffer 1345. This buffer 1345 produces abuffered internal clock signal CLKB[i,j] on bus 1347 that is at adifferent phase than the CLK[i,j] signal. This CLKB[i,j] signal is usedto transmit on signal set 1349 to the DQ bus 1325, to receive from theRQ and DQ buses, and to perform all other internal operations in thememory component. Because there is a single clock domain (i.e.,CLKB[i,j]) inside the memory component 1310, there is no need for clockdomain crossing logic in the memory component as there was in system PA1(of FIG. 2). In addition, calibration logic 1350 (“M_(CAL)”) has beenadded to the memory component 1310. In system 1300, this logic 1350 isused in conjunction with the calibration logic 1355 (“C_(CAL)”) that hasbeen added to the controller 1305.

Because the internal clock CLKB[i,j] of the memory component runs atfour times the rate of the RQ[i,j] bus 1352, it is possible to usesampling logic 1360 within the memory to adjust for unknown skew betweenthe internal clock signal CLKB[i,j] on bus 1347 and the bit signals onRQ interface line 1352 that is caused by the buffer delay t_(Bij).

As in the controller of system 400, write data D[0,j] transmit logic andthe read data Q[0,j] receive logic must be operated in two differentclock domains (CLKD[0,j] and CLKQ[0,j]) that have different phases thanthe CLK1 domain that the rest of the controller uses. As a result, thereis phase adjustment logic 1365 and 1368 that delays CLK1 by t_(Dij) andt_(Qij), respectively, to form the CLKD[0,j] and CLKQ[0,j] signals,respectively. Because of the multipliers 1335, CLKD[0,j] and CLKQ[0,j]are also four times the frequency of CLK1.

As in system 400, the values t_(Dij) and t_(Qij) are functions of thepropagation delay parameters t_(PROPCLKij), t_(PROPDij), andt_(PROPQij). These propagation delay parameters are relativelyinsensitive to temperature and supply voltage changes. Values t_(Dij)and t_(Qij) are also a function of the delay t_(Bij) of the memorycomponent's clock buffer 1345 as well as other delays such as thoseassociated with transmit and receive circuits. The clock buffer delaywill change during system operation because it is relatively sensitiveto temperature and supply voltage changes. The programmable valuest_(Dij) and t_(Qij) that are generated during system initialization aredynamically updated during system operation, and a calibration process(using the calibration logic M_(CAL) and C_(CAL)) is needed to keep thevalues updated.

In system 1300, enhancements have been made to the clock domain crossinglogic 1380 so that the calibration process is handled completely byhardware. Such enhancements are useful to insure that the performance ofthe system is not significantly impacted by the overhead of thecalibration process.

Some of the important differences between system 300 (FIG. 4) and system1300 are listed below:

(1) System 1300 has a clock buffer 1345 (which can have variable delayduring system operation), rather than a PLL/DLL clock recovery circuiton the memory component ;

(2) System 1300 has calibration logic 1350 and 1355 to the memorycomponent and controller, respectively, to support a calibrationprocess;

(3) System 1300 has enhanced clock domain crossing logic 1380 to improvethe efficiency of the calibration process;

(4) Signal transmission on the RQ bus 1315 may be at a lower frequencythan the CLK and DQ buses (one fourth the rate of the CLK1 signal inthis example); and

(5) System 1300 includes sampling logic 1360 in the memory component forthe RQ bus.

FIG. 14A shows the timing of a read transfer for system 1300. As noted,the controller 1305 uses an internal clock signal CLK1, 1330, for itsinternal operations. Rising edge 0, 1410, of the CLK1 signal 14(a)samples the signal on the internal RQ_(C) bus 1385 with a register andcauses the register to drive the sampled signal value onto the RQ[i,0]bus 1315 after the delay t_(V,RQ). This delay is the output valid delay(the clock-to-output delay) of the register and driver that samples RQand drives it out of the controller. The address and control informationassociated with the read command is denoted by the label “READ” in thefigure. The RQ[i,0] signals on the RQ bus 1315 propagate to memorycomponent [i,j] after a propagation delay t_(PROP,RQij) to become theRQ[i,j] signals, where they are received by the memory component. Thesetup time of the signal on bus RQ[i,j] is t_(S,RQ), measured to therising edge 1455 of CLKB[i,j] that causes sampling to be performed bythe sampling logic 1360.

The CLK1 signal is frequency multiplied, here by four, to give theCLK[i,0] signal 14(d), which is delayed by t_(V,CLK) with respect toCLK1. This delay is the output valid delay of the driver that drivesCLK[i,0]. The CLK[i,0] signal propagates to memory component [i,j] aftera propagation delay t_(PROP,CLKij) to become signal CLK[i,j] 14(e),where it is received by the memory component 1310 and buffered by buffer1345 to become the internal clock signal CLKB[i,j] after a delayt_(Bij).

Here, because there are four CLKB[i,j] cycles for each bit received oneach signal of set RQ[i,j], there is freedom to choose one of the fourrising edges to do the sampling. This freedom is necessary because thedelay t_(Bij) will be different between the memory components in rank[i], and the optimal sampling point will need to be separately adjusted.This adjustment is accomplished by selecting one of the four CLKB[i,j]rising edges to receive the RQ[i,j] bus. The sampling edge is denoted bythe heavy arrow at 1455-1458 on one of every four of the rising edges ofCLKB[i,j] 14(f). Here, this sampling edge is also used for internaloperations in the memory component. Note that all four CLKB[i,j] risingedges are used to receive data from the D[i,j] signal set and totransmit data onto the Q[i,j] signal set. The bit time in this exampleis equal to the CLKB cycle time (which is also the CLK4 cycle time sincethese two clock signals are frequency locked). The parametert_(SAMPLEij) accounts for the delay due to the need to choose one of thefour CLKB[i,j] rising edges for sampling. t_(SAMPLEij) is measured ordenoted in integral units of t_(CLK4CYCLE), the cycle time of CLKB[i,j].Because CLKB[i,j] is periodic, t_(SAMPLEij) may be positive or negative,and this clock accounts for the time needed to make equation (1)correct:t _(V,RQ) +t _(PROP,RQij) +t _(S,RQ) =t _(V,CLK) +t _(PROP,CLKij) +t_(Bij) +t _(SAMPLEij)  (1)

The details of the calibration process used to select the sampling edgewill be described later.

Once the RQ[i,0] bus has been sampled (denoted by the large black circle1433 in the figure), the internal read access t_(CAC,INT) is started. Inthis example, this internal read access requires a total of3*t_(CLK1CYCLE) (which is equivalent 12*t_(CLK4CYCLE)).

An external read access delay t_(CAC,EXT) may also be defined. Thisdelay is the time from the CLK[i,j] clock signal rising edge whicheffectively samples the RQ[i,j] bus to the time that the first bitbecomes valid on the Q[i,j] signal set:t _(CAC,EXT) =t _(Bij) +t _(SAMPLEij) +t _(CAC,INT) +t _(V,Q)  (2)

A second external read access delay t_(CAC,EXT2) (not shown) may also bedefined. The delay is from the time a signal on the RQ[i,j] bus is setup to the time the first bit becomes valid on the Q[i,j] signal set:t _(CACEXT2) =t _(S,RQ) +t _(CAC,INT) +t _(V,Q)

The external read access delay (t_(CAC,EXT)) is a useful delay parameterbecause it includes all delay terms contributed by the memory component1310, but none contributed by the external interconnections or by thecontroller. Equation (2) includes two terms (t_(Bij) and t_(V,Q)) thatwill change continuously due to, for example, temperature and supplyvoltage variations during system operation. In contrast, the internalread access delay term t_(CAC,INT), shown graphically in 14(c), willremain constant during system operation. The term t_(SAMPLEij) willchange in increments of t_(CLK4CYCLE) because of sampling logic changesthat compensate coarsely for some temperature and supply voltagevariations during system operation. Likewise, the second external accessdelay (t_(CAC,EXT2)) includes the terms t_(S,RQ) and t_(V,Q) that changeduring system operation.

As a result, the external read access delay t_(CAC,EXT) (ort_(CAC,EXT2)) of the memory component will change during systemoperation. This change (plus any changes contributed by the externalinterconnections or by the controller) can be compensated for using, forexample, an adjustable timing value t_(PHASERj) in the controller. Dueto the ability of the present invention to “calibrate out” largevariations in the external access time of a memory component over time,the difference in external access time between two similar memory readoperations (one at a first time and another at a later time when thetemperature and/or voltage of the memory component has changed), or twosimilar memory write operations, may exceed a half-symbol time interval.Two memory operations are “similar” for purposes of this discussion ifthey have the same internal access time, or if they have very similarinternal access times (e.g., which differ by less than a multiplicativefactor of 1.1). For instance, two read access operations that are both“page hits” will typically be similar memory read operations having thesame internal access time, while a read access that is a page hit andanother read access that is a page miss will typically have verydifferent internal access times and thus would not be similar memoryoperations. Two memory requests (whether read requests or writerequests) are “similar” for purposes of this discussion if the resultingmemory operations have the same or similar internal access times. Also,as noted earlier in this document, the “symbol time interval” is theduration of an average symbol on the DQ bus as measured at the memoryinterface, and is sometimes called a “bit time interval.”

In a preferred embodiment, the timing compensation capabilities of thecalibration circuitry are sufficiently large that the difference inexternal read access time between two similar memory read operations, ortwo similar memory write operations, can exceed a full symbol timeinterval.

At the end of the t_(CAC,INT) interval, the four bits of read dataQc[3:0] in 14(g) from the memory core are sampled, using the samplingedge of CLKB[i,j]. The four bits are driven from the memory componentserially, after the delay t_(V,Q). This delay is the output valid delay(the clock-to-output delay) of the register and driver that samplesQc[3:0] and drives it out of the memory component onto the Q[i,j] signalset 1349.

The Q[i,j] signal of 14(h) propagates to the controller after apropagation delay t_(PROP,Qij) to become signal Q[0,j], where it isreceived by the controller. The setup time of signal Q[0,j] is t_(S,Q),measured to the rising edge of internal clock signal CLKQ[0,j] as shownin 14(i). The four serial bits are converted to parallel form after thedelay t_(StoP,Q) (this delay is equivalent to 1*t_(CLK1CYCLE) or4*t_(CLK4CYCLE)). The internal clock CLKQ[0,j] is delayed from CLK1 by(t_(OFFSETR)+t_(PHASERj)). t_(OFFSETR) is a fixed offset of4*t_(CLK1CYCLE) in this example. t_(PHASERj) is an adjustable delay foreach slice [j]. The delay is updated and adjusted through a calibrationprocess so that it remains centered on the data window of the bits beingreceived on the Q[0,j] bus. The details of this calibration process willbe described in a later section. The value of t_(PHASERj) shown at 14(k)is preferably chosen to satisfy equation (3):t _(V,CLK) +t _(PROP,CLKij) +t _(Bij) +t _(SAMPLEij) +t _(CAC,INT) +t_(V,Q) +t _(PROP,Qij) +t _(S,Q) +t _(StoP,Q) =t _(OFFSETR) +t_(PHASERj)  (3)

Many of the terms in Eqn. (3) will be affected by temperature and supplyvoltage variations during system operation. Here, t_(PHASERj) will beadjusted to compensate for these variations. t_(PHASERj) can be adjustedthrough a range of t_(RANGER). The value of t_(RANGER) has a value of4*t_(CLK1CYCLE) in this embodiment. The range of t_(RANGER) is chosen toaccommodate t_(PHASERj) regardless of whether the terms in Eqn. (3)assume their minimum or maximum values.

Because each slice of memory components can have a differentt_(OFFSETR)+t_(PHASERj) value within a rank of memory components, itbecomes necessary for the controller to add some variable delay toensure that the read data Qc[3:0] becomes available at a fixed time. Thefixed time that is chosen in this example is t_(OFFSETR)+t_(RANGER), andhas a value of 8*t_(CLK1CYCLE). Stated differently, read data from theread command sampled on CLK1 edge 0 is available for all slices on CLK1edge 8.

The compensating delays are inserted by the controller's domain crossinglogic 1380. The delays are t_(SKIPRj)+t_(LEVELRj). t_(SKIPRj) is theterm that inserts a delay that is a fraction of t_(CLK1CYCLE).t_(LEVELRj) is the term that inserts a delay that is an integer multipleof t_(CLK1CYCLE) of signal 14(a), where the integer multiple is equal toor greater than zero.

The propagation delay t_(PROP,Qij) for data signals and the propagationdelay t_(PROP,CLKij) for clock signals remain substantially constant,even with changes in temperature and voltage levels. As a result,differences in the external access time of memory components in the samerank are almost completely the result of differences in the internaloperating characteristics of the memory components, which in turn aredue to manufacturing differences as well as differences in temperatureand voltage. In prior art systems, the external access time of all thememory components in a single rank would have to be substantially thesame, within a tolerance of much less than half a symbol time interval,in order to avoid data transmission errors. In contrast, the calibrationcircuitry of the present invention enables the use of memory componentsin the same rank of a system that have external access times, forsimilar memory requests for similar memory operations, that differ bymore than half of a symbol time interval. The calibration circuitry ofthe present invention can handle such large differences in externalaccess time because a respective access compensation time is separatelydetermined for each memory component of the system. Further, because thecompensation time value that is determined for each memory component hassuch a large range of possible values, external access time differences(for memory components in the same rank of the system) greater than afull symbol time interval can be easily compensated, and thus“calibrated out” of the system.

Referring to FIG. 14B, another way to distinguish synchronous and staticmesochronous systems from dynamic mesochronous systems is to look at thealignment of the data bit window with respect to the clock signal at thepins of the component. For example, in FIG. 14A, the clock signal 14(e)CLK[i,j] received at the memory component may be compared to the readdata 14(h) Q[i,j] output by the memory component. In a synchronous orstatic mesochronous system, the range of the relative phase of thesesignals (the drive offset time) will be essentially fixed. In thisexample, the bit time for Q[i,j] starts at a point −90 degrees in theCLK[i,j] cycle and is equal to a CLK[i,j] cycle time. The conventionused here is to measure the phase offset (delay offset) from thebeginning of a bit time to the rising CLK[i,j] edge that is associatedwith that bit time. In a dynamic mesochronous system, the range of therelative phase of these signals can be expected to vary over a full bittime (plus or minus a one-half of a symbol time interval or plus orminus 180 degrees).

For example, as shown in FIG. 14B, the phase difference could bemeasured at different times during system operation. For a staticmesochronous system, the relative phase values stay within a narrowrange (plus or minus 20 degrees, in this case) around the nominal phaseoffset of −90 degrees. For a dynamic mesochronous system, the relativephase values can vary over the maximum possible range (plus 90 or minus270 degrees in this example) around the nominal phase offset of −90degrees.

This provides another way, then, to distinguish the types of systems. Ifthe relative phase of the clock signal and the data signal remain withina range of plus or minus 90 degrees (plus or minus one-quarter of asymbol time interval) from the nominal operating point during systemoperation, then the system is a synchronous or static mesochronoussystem. If relative phase of the clock signal and the data signal variesover a range that exceeds plus or minus 90 degrees (plus or minusone-quarter of a symbol time interval), then the system is a dynamicmesochronous system.

This means of discriminating static mesochronous and dynamicmesochronous systems can be extended to systems in which there are twoor more bit times per clock cycle. In this case, the relative phase ismeasured between a clock event (the rising edge in this example) and thebeginning of bit time that straddles or is otherwise associated with theclock event. 360 degrees of phase are equal to the bit time interval(the shorter of the bit time and clock cycle intervals). In a staticmesochronous system, the relative phase of the rising edge and the startof the associated bit time remain within a range of plus or minus 90degrees from the nominal phase offset. In a dynamic mesochronous system,the relative phase of the clock event and the start of the associatedbit time can drift outside this range of plus or minus 90 degrees fromthe nominal phase offset. Note that since there are a set of two or morebit times associated with each clock event, it is necessary toconsistently use the same bit time from each set when evaluating thephase shift during system operation.

This means can also be extended to systems in which there are two ormore clock cycles per bit time. In this case, the relative phase ismeasured between a clock event (e.g., the rising edge of the clocksignal) and the beginning of the bit time that straddles or is otherwiseassociated with the clock event. 360 degrees of phase are equal to theclock cycle interval (the shorter of the bit time and clock cycleintervals). In a static mesochronous system, the relative phase of theclock event and the start of the associated bit time remain within arange of plus or minus 90 degrees from the nominal phase offset. In adynamic mesochronous system, the relative phase of the clock event andthe start of the associated bit time can drift outside this range ofplus or minus 90 degrees from the nominal phase offset. Note that thereare a set of two or more clock cycles associated with each bit time, andtherefore it is necessary to consistently use the same clock event fromeach set when evaluating the phase shift during system operation.

FIG. 15 shows the timing of a write transfer for system 1300. As noted,the controller 1305 uses an internal clock signal CLK1 for its internaloperations. Rising edge 0, 1510 of the CLK1 signal in 15(a) samples thesignal on internal RQ_(C) bus with a register and causes the register todrive the sampled signal value onto the RQ[i,0] bus after the delayt_(V,RQ). This delay is the output valid delay (the clock-to-outputdelay) of the register and driver that samples RQ_(C) and drives it outof the controller. The address and control information associated withthe write command is denoted by the label “WRITE” in the figure. TheRQ[i,0] bus propagates to memory component [i,j] after a propagationdelay t_(PROP,RQij) 1530 to become the RQ[i,j] signal of 15(c), which isreceived by the memory component: [i,j]. The setup time of bus RQ[i,j]is t_(S,RQ) 1535, measured to the rising edge 1555 of CLKB[i,j], shownin 15(f), that performs the sampling.

The CLK1 signal is multiplied in frequency, here by four, to give theCLK[i,0] signal of 15(d), which is delayed by t_(V,CLK) 1540 relative tothe CLK1 signal. This delay is the output valid delay of the driver thatdrives CLK[i,0] out of the controller. The CLK[i,0] signal propagates tomemory component [i,j] after a propagation delay t_(PROP,CLKij) 1545 tobecome signal CLK[i,j] of 15(e), where it is received by the memorycomponent and buffered to become the internal clock signal CLKB[i,j]after a delay t_(Bij), 1550.

Here, because there are four CLKB[i,j] cycles for each bit received fromeach signal of set RQ[i,j], there is freedom to choose one of the fourrising edges to do the sampling. This freedom is necessary because thedelay t_(Bij) will be different between the memory components in rank[i], and the optimal sampling point will need to be separately adjusted.This adjustment is accomplished by selecting one of the four CLKB[i,j]rising edges to receive the RQ[i,j] bus. The sampling edge is denoted bythe heavy arrow at 1555-1570 on one of every four of the rising edges ofCLKB[i,j] in 15(f). This sampling edge is also used for all internaloperations in the memory component. Note that all four CLKB[i,j] risingedges are used to receive signals on the D[i,j] signal set and totransmit signals on the Q[i,j] signal set. The parameter t_(SAMPLEij)1585 in 15(f) accounts for the delay due to the need to choose one ofthe four CLKB[i,j] rising edges for sampling. The duration oft_(SAMPLEij) is measured in integral units of t_(CLK4CYCLE), the cycletime of CLKB[i,j]. Because CLKB[i,j] is periodic, t_(SAMPLEij) may bepositive or negative; it accounts for the time needed to make thefollowing equation correct:t _(V,RQ) +t _(PROP,RQij) +t _(S,RQ) =t _(V,CLK) +t _(PROP,CLKij) +t_(Bij) +t _(SAMPLEij)  (4)

The details of the process to select the sampling edge will be describedlater.

The steps of the write transfer described above are virtually identicalto the corresponding steps of the read transfer. However, particulardifferences between the read and write transfer exist.

Once the RQ[i,0] bus has been sampled (denoted by the large black circle1533 in the figure), the internal write access time interval t_(CWD,INT)is started. This requires a total of 3*t_(CLK1CYCLE) (which isequivalent 12*t_(CLK4CYCLE)) in this example.

An external write access delay t_(CWD,EXT) 1575 may also be defined.This delay 1575 is the time from the CLK[i,j] clock signal rising edgewhich effectively samples the signal on the RQ[i,j] bus to the time thatthe first bit is set up on the D[i,j] signal set:t _(CWD,EXT) =t _(Bij) +t _(SAMPLEij) +t _(CWD,INT) −t _(S,D) −t_(StoP,D)  (5)

A second external write access delay t_(CWD,EXT2) (not shown) may bedefined. This delay is from the time a signal on the RQ[i,j] bus is setup to the time the first bit is set up on the D[i,j] signal set:t _(CWD,EXT2) =t _(S,RQ) +t _(CWD,INT) −t _(S,D) −t _(StoP,D)

Eqn. (5) is useful because it includes all delay terms contributed bythe memory component, but none contributed by the externalinterconnections or by the controller. Eqn. (5) includes two terms(t_(Bij) 1550 and t_(S,D) 1578) that will change continuously due totemperature and supply voltage variations during system operation. Incontrast, the terms t_(CWD,INT) and t_(StoP,D) 1580 will remain constantduring system operation. The term t_(SAMPLEij) 1585 will change inincrements of t_(CLK4CYCLE) because of sampling logic changes thatcompensate coarsely for some temperature and supply voltage variationsduring system operation. Likewise, the second external access delay(t_(CWD,EXT2)) includes the terms t_(S,RQ) and t_(S,D) that changeduring system operation.

As a result, the external write access delay t_(CWD,EXT) 1575 of thememory component will change during system operation. This change (plusany changes contributed by the external interconnections or by thecontroller) will be compensated with an adjustable timing valuet_(PHASETj) in the controller.

At the end of this t_(CWD,INT) interval, shown graphically in 15(c), thefour bits of write data D_(M)[3:0] in 15(g) are held in a register andare available for writing to the memory core after the delay t_(V,D).This delay 1590 is the output valid delay (the clock-to-output delay) ofthe holding register.

The D[0,j] signals propagate to the memory component after a propagationdelay t_(PROP,Dij) to become the D[i,j] signals of 15(h), which arereceived by the memory component. The setup time of signal set D[i,j] ist_(S,D) 1578, measured to the rising edge of internal clock signalCLKB[i,j], here measured to rising edge 1565. The four bits are receivedby the memory component serially, after the delay t_(StoP,D). This delay1580 is the serial-to-parallel conversion delay (this is equivalent to1*t_(CLK1CYCLE) or 4*t_(CLK4CYCLE)). The four bits of write data becomevalid a time t_(V,D) after the last of these four bits is sampled by therising edge of internal clock CLKB[i,j]. This delay 1590 is the outputvalid delay (the clock-to-output delay) of the register and on thecontroller.

The internal clock CLKD[0,j] of 15(j) is delayed from CLK1 by(t_(OFFSETT)+t_(PHASETj)). Here, t_(OFFSETT) 1588 is a fixed offset of1*t_(CLK1CYCLE). t_(PHASETj) is an adjustable delay for each slice [j].This adjustable delay is updated through a calibration process involvingcalibration logic 1350 and 1355 (FIG. 13) that keeps the write data bitson the bus carrying the D[i,j] signal set centered with respect to theCLKB[i,j] clock signal that is sampling them in the memory component[i,j]. The details of this calibration process will be described later.The value of t_(PHASETj), shown in 15(k), is preferably chosen to makethe following equation correct:t _(V,CLK) +t _(PROP,CLKij) +t _(Bij) +t _(SAMPLEij) +t _(CWD,INT) =t_(OFFSETT) +t _(PHASETj) +t _(VD) +t _(PROP,Dij) +t _(S,D) +t_(StoP,D)  (6)

Many of the terms in Eqn. (6) will be affected by temperature and supplyvoltage variations during system operation. t_(PHASETj) is adjusted bythe calibration process to compensate for these variations. t_(PHASETj)can be adjusted through a range of t_(RANGET) 1586. Here, t_(RANGET) hasa value of 4*t_(CLK1CYCLE). This range is chosen to accommodatet_(PHASETj), regardless of whether the terms in the above equationassume their minimum or maximum values.

Each slice of memory component can have a differentt_(OFFSETT)+t_(PHASETj) value within a rank of memory components.However, each memory component will be presented with write data at itscore at the appropriate time (t_(CWD,INT) after the CLKB[i,j] clock edgethat samples the RQ[i,j] bus).

The t_(PHASETj) delay 1592 is inserted by the controller's domaincrossing logic 1380. Other delays inserted includet_(SKIPTj)+t_(LEVELTj). t_(SKIPTj) 1595 is a delay that is a fraction ofT_(CLK1CYCLE). t_(LEVELTj) 1598 is a delay that is an integer multipleof t_(CLK1CYCLE) of the signal in 15(a).

FIG. 16 shows the logic for the memory component 1600 at position [i,j]in system 1300. There are three buses that connect the memory componentto the external system: CLK[i,j], RQ[i,j] and DQ[i,j]. In this example,the RQ[i,j] bus 1604 has N_(RQ) signals, where N is an integer greaterthan zero, and the other two buses have one signal each.

As depicted in FIG. 16, memory component 1600 is configured to connectto the controller with one DQ wire per slice. Other embodiments couldconnect the memory component to the controller with more than oneDQ[i,j] signal by a simple extension of the methods described for system1300.

Memory component 1600 has three internal logic blocks forming the memoryinterface: M1, M2, and M3. There is a memory core (block M5) thatcontains storage cells (i.e., the main memory array subcomponent of thememory component). There is also a set of registers and multiplexinglogic (block M4) that form the calibration logic (also called M_(CAL)earlier) for the memory component 1600.

Block M1 receives the CLK[i,j] and RQ[i,j] buses 1602 and 1604,respectively. Block M1 produces a buffered clock CLKB[i,j] that is usedthroughout memory component 1600. Block M1 also produces a Load signalon bus 1608 that indicates which CLKB[i,j] signal edges are used forinternal operations. A Commands bus 1610 carries command signals thatindicate which memory command (READ, WRITE, WRPAT0, WRPAT1, RDPAT0,RDPAT1, etc.), if any, is being executed.

Block M2 transmits read data on the DQ[i,j] bus 1612. Block M2 performsa parallel to serial conversion on data bits received via the busQ_(M)[3:0] 1614 from Block M4. Block M2 also uses the buffered clockCLKB[i,j] and Load signals.

Block M3 receives write data on the DQ[i,j] 1612 bus. Block M2 performsa serial to parallel conversion and outputs the resulting bits on busD_(M)[3:0] 1620. Block M2 also uses the buffered clock CLKB[i,j] on bus1606 and Load signals on bus 1608.

The calibration logic M4 consists of two registers PAT0 and PAT1 1630and 1635, respectively, which can be loaded with write data on busD_(M)[3:0] 1620. The loading of the write data occurs when the WRPAT0 orWRPAT1 commands are asserted on the Commands bus 1610, causing the C2 orC1 load signals on buses 1640 and 1645, respectively, to be asserted.

The calibration logic M4 is also able output the contents of the tworegisters PAT0 and PAT1 onto the bus Q_(M)[3:0] 1614 instead of the readdata Q_(MO)[3:0] 1650 from the memory core, block M5. The contents ofthe registers PAT0 and PAT1 are output onto the bus Q_(M)[3:0] 1614 whenread commands RDPAT0 and RDPAT1, respectively, are received by thememory component via the RQ bus. These read commands cause the C4 selectsignal 1655 to be asserted and the C3 signal 1660 to be deasserted orasserted, respectively, so as to route the data from the PAT0 and PAT1registers to the Q_(M)[3:0] bus 1614.

The two “pattern” registers 1630 and 1635 assume specific values (i.e.,are automatically initialized) when the memory component 1600 is firstpowered up. In one embodiment, the pattern registers are initialized toa predefined value (e.g., “0 1 0 0”) by circuitry that detects theramping-up of the supply voltage. In another embodiment, the registerinitialization circuits is responsive to a RESET command on the commandbus 1610 or to a sideband signal that causes the memory component 1600to reset to a known state (this signal is not shown). Initialing thepattern registers 1630, 1635 to a known value is important for correctinitial execution of the calibration process. These initial values couldbe replaced by other values later.

FIG. 17 shows the logic for block M1 of the memory component 1600 atposition M[i,j] in system 1300 for producing buffered clock signalCLKB[i,j] on bus 1606, Load signal on bus 1608/1715 and Commands signalson bus 1610. More generally, the logic diagram in FIG. 17 and the timingdiagram in FIG. 18 show how the calibration apparatus of block M1 isconfigured to determine the suitability of a plurality of timing events(i.e., each of the “1”s on the RQ[i,j][b] signal after a CALSET commandis received on the RQ[i,j] signal) and to select, based on thesuitability determination, one of the plurality of timing events for useas a sampling point for sampling the symbols on the RQ[i,j] signal. Inan alternate embodiment, similar calibration circuitry to that used inM1 could be provided to determine the suitability of a plurality oftiming events for use as a driving point for driving symbols onto asignal, and to select, based on the suitability determination, one ofthe plurality of timing signals for use as the driving point.

It should be noted that the calibration apparatus in block M1 of eachmemory component operates independently of the calibration apparatus inblock M1 of each other memory component in the memory system. Thus, evenif the same CALSET and CALTRIG commands are sent simultaneously tomultiple memory components, each memory component will independentlyselect the best (i.e., most suitable) timing event for sampling theRQ[i,j] signal. As a result, two memory components in the same rank of amemory array may select different timing events at which to sample theRQ[i,j] signal. The same independence of the timing event selectionwould also apply to systems in which calibration logic is used to selectthe most suitable timing event (e.g., clock edge) for use as a drivingpoint for driving symbols onto a signal.

System 1700 receives the CLK[i,j] and RQ[i,j] buses 1602 and 1604,respectively. The buffered clock CLKB[i,j] signals produced by buffer1710 are used by the rest of the memory component. The register 1712produces a Load signal on bus 1715/1608 which indicates which edges ofCLKB[i,j] are to be used for internal operations. A Commands bus 1610carries command signals that indicate which memory command (READ, WRITE,etc.) is being executed.

The clock signal CLK[i,j] is buffered to produce a buffered CLKB[i,j]signal that clocks a set of register bits, here six bits, which producethe signal Load on buses 1608 and 1715. The six register bits are calledLoad, CalState[1:0], CalForm[1:0] and CalEn. The CalState[1:0] register1717 counts through four states {00, 01, 10, 11}. The CalForm[1:0]register 1720 contains a two bit value that is compared to CalState[1:0]bits in each cycle. When the bits from the CalState[1:0] register 1717match the bits of the CalForm[1:0] register 1720, a Load signal isasserted by the Load register 1712 in the next CLKB[i,j] cycle on theLoad bus 1608.

The CalEn register 1725 is used to update the value held in register1720. Register 1725 is responsive to two signals, CALTRIG 1730 andCALSET 1735, which are commands decoded from bus 1604 by decode logic1722. The use of these two signals will be further described relative tothe timing diagram for system 1700.

FIG. 18 shows the timing for block M1 of the memory component 1600. Tofacilitate unambiguous references to signals in various timing diagramsof this documents, signals denoted as (a), (b) and so on in FIG. 18shall be denoted as signals 18(a), 18(b) and so on in the text of thisdocument. FIG. 18 shows the sequence needed to generate load signals andupdate the CalForm[1:0] value, signal 18(k), to accommodate any timingshifts due to temperature and supply voltage changes during systemoperation. It should be noted that all the RQ signals shown in FIG. 18are signals generated by the controller and sent to the memory componentwhose operations are depicted in FIG. 18.

The clock signal CLK[i,j] (on bus 1602 in FIG. 16) is shown as waveform18(a). Clock signal CLK[i,j] is buffered and delayed by t_(Bij) toproduce CLKB[i,j], waveform 18(b). The rising edges of the CLKB[i,j]signal are numbered to label the timing events. The large black circlesindicate the sampling point of signals by registers clocked byCLKB[i,j].

The RQ[i,j] bus 1604 carries the N_(RQ) signals labeledRQ[i,j][N_(RQ)-1:0]. These signals are shown as waveform 18(c), alongwith signal RQ[i,j][b] broken out individually below as waveform 18(d).Note that index “b” is within the range [N_(RQ)-1:0] for this example.Signals 18(c) and 18(d) are used to encode three commands in system 1700when updating sequences: CALSET (calibration set), NOP (no operation),and CALTRIG (calibration trigger). The label “any” on these signalsindicates any other command may be provided during the respectiveinterval. Signal RQ[i,j][b] must be low for the NOP command and theCALSET command 18(e), and must be high for the CALTRIG command signal18(f). Other restrictions on the command encoding are not necessary.

The update sequence begins with the CalState[1:0] register 1717incrementing via incrementer 1740 through its four possible states. TheCalState signal is shown as signal 18(i), and the incremented signal isrepresented as waveform 18(j). In the example shown in FIG. 18, theCalFrm[1:0] register 1720 holds the value “00”, and therefore thecomparator 1742 finds a match during the cycles in which the value inthe CalState[1:0] register is “00”. The positive output of thecomparator results in a “1” being stored in the Load register 1710 atthe next positive going edge of the CLKB[i,j] signal, at which time thevalue in the CalState[1:0] register becomes “01”. Signal 18(k) depictsthe signal CalForm stored in register 1720, and signal 18(l) depicts theLoad signal waveform. In other words, the Load signal waveform 18(l) isequal to “1” in each clock cycle that follows a clock cycle in which thevalue in the CalState[1:0] register 1717 equals the value in theCalForm[1:0] register 1720.

The RQ[i,j][N_(RQ)-1:0] bus 1604 is sampled on edge 1 (because the Loadsignal 18(l) is asserted during edge 1) and is decoded as the CALSETcommand, causing the CALSET signal 18(e) to be asserted. Signal 18(e) issampled by the CalEn register 1725 on edge 2, causing the CalEn signal18(h) to be asserted after edge 2.

The RQ[i,j][N_(RQ)-1:0] bus is sampled again on edge 5, and is decodedas a NOP and ignored.

The RQ[i,j][N_(RQ)-1:0] bus is sampled again on edge 9, and is decodedas a CALTRIG command, which is ignored and treated the same as a NOP.However, the RQ[i,j][b] signal is asserted and sampled high on edges 9,10, 11, and 12. A set of three registers 1745, 1750, 1755 and an “AND”gate 1760 detect three high assertions in a row (of the RQ[i,j][b]signal) and assert the CALTRIG signal 18(f) as indicated by arrow 1810.Signal 18(f) causes the CalClr signal, 18(g), to be asserted. The CalClrsignal 18(g), in turn, is sampled by the CalEn register 1725 (indicatedby arrow 1830), causing it to go low (i.e., be reset) after edge 12. TheCalClr signal 18(g) also enables the CalForm[1:0] register to load theincremented value of the CalState[10] register (as indicated by arrow1840), and to output its new value after edge 12. This new value is“01”, meaning that the Load signal on bus 1608 will now be assertedduring the cycles in which the CalState[1:0] register 1717 is “10”. Inother words, the sampling point selected by the Load register 1765 hasshifted right by one CLKB[i,j] cycle.

The RQ[i,j][N_(RQ)-1:0] bus is sampled on edges 13 and 14, and isdecoded as a NOP and ignored. The RQ[i,j][N_(RQ)-1:0] bus is sampledagain on edge 18, and is decoded as a valid command, and the command isexecuted.

The timing relationship in FIG. 18 depicts a simple hardware implementedalgorithm that searches for a string of three sampled “1”s on theRQ[i,j][b] signal and updates the CalForm[1:0] value to the value in theCalState[1:0] register plus 1. This CalForm[1:0] value is the one thatmakes the Load signal assert during the second sampled “1”. The previousvalue of CalForm[1:0] caused the Load signal to assert during the firstsampled “1”, which is not optimal because there is less timing margin.The Load signal controls not only when the command signalRQ[i,j][N_(RQ)-1:0] is sampled and decoded, but also controls the timingof data loads in the M2 and M3 blocks and in PAT0 and PAT1 registers.

FIG. 19A shows the logic for block M2 of the memory component 1600 atposition [i,j] in system 1300. Block M2 performs a parallel to serialconversion, taking four parallel bits of read data from the Q_(M)[3:0]bus 1614 and serially transmitting the read data onto the bi-directionalDQ[i,j] bus 1612. Block M2 also uses the buffered clock CLKB[i,j] andLoad signals.

The Load signal on bus 1608 is asserted during one of every four risingedges of CLKB[i,j]. In this example, the edge of CLKB[i,j] that isselected is the same as the one that is used by block M1 to receive theRQ[i,j] signal. As a result, the internal read access time t_(CAC,INT)will be an integral multiple of t_(CLK1CYCLE) (3*t_(CLK1CYCLE) or12*t_(CLK4CYCLE)). Other embodiments could deliberately misalign theLoad signal for receiving the RQ[i,j] signal on bus 1604 and the Loadsignal for transmitting the DQ[i,j] signal 1612 to match a timingrequirement of the memory core 1680.

Register 1930 is loaded with four bits of information from theQ_(M)[3:0] bus 1614 during each clock cycle, but only the informationloaded in the clock cycle prior to each Load signal is used. When theLoad signal 18(l), or 1608, is asserted, the four bits of information inregister 1930 are steered through multiplexer 1910 to the four one-bitregisters 1920 and are loaded into those registers 1920 upon the clockedge that occurs while Load is enabled. The outputs of a last one of theregisters 1920 is asserted as the DQ[i,j] signal after the clock edge.On the next three clock edges the multiplexer shifts the remaining threebits onto the DQ[i,j] signal.

FIG. 19B shows the logic for block M3 of the memory component 1600 atposition [i,j] in system 1300. Block M3 receives write data on theDQ[i,j] bus 1612. A serial to parallel conversion is performed byregisters 1940 and multiplexer 1950 to create the parallel data assertedon bus D_(M)[3:0] 1620. The serially connected registers 1940 areclocked by the buffered clock signal 1606, and Load signal transfers thecontent of the registers 1940 through the multiplexer 1950 to register1960.

The Load signal on bus 1608 is asserted on one of every four risingedges of the CLKB[i,j] signal on bus 1606. In this example, the selectededge is the same edge as the one that is used by block M1 for receivingthe RQ[i,j] signal on bus 1604. As a result, the internal write accesstime t_(CWD,INT) will be an integral multiple of t_(CLK1CYCLE)(1*t_(CLK1CYCLE) or 4*t_(CLK4CYCLE)). Other embodiments coulddeliberately misalign the Load for receiving the RQ bus and the Loadsignal for receiving the DQ bus 1612 to match a timing requirement ofthe memory core 1680.

During a write transfer, the four one-bit registers 1940 connectedserially to the DQ[i,j] signal 1612 continuously shift in the write datathat is present on each rising edge of CLKB[i,j]. When the Load signalon bus 1608 is asserted, the most recent shifted-in write data is loadedin parallel to the register 1960 connected to the D_(M)[3:0] bus 1620.When the Load signal is deasserted, the contents of register 1960 arerecirculated through the multiplexer on line 1970.

FIG. 20 shows the logic 2000 for the controller component 1305 in thesystem 1300. There are three buses that connect the controller to thememory components of the memory system: CLK[i,0] 1320, RQ[i,0] 1315 andDQ[0,j] 1325. Logic 2000 is made up of three blocks: C1, C2 and C3.Block C1 contains clock generator circuitry. Block C2 contains circuitryfor each memory rank [i] and connects to the N_(RQ) signals of the bus1315 and the one CLK[i,0] signal 1320. Block C3 contains circuitry foreach memory slice [j] and connects to the one signal of the DQ[0,j] bus1325. The controller of FIG. 20 will typically contain other blocks ofcircuitry, some or all of which are not part of the memory interface,but these blocks are not shown here.

Logic 2000 assumes that each memory component slice connects to thecontroller with one DQ signal. Other embodiments could connect thememory component to the controller with more than one DQ signal by asimple extension of the methods described for system 1300.

There are six sets of signals that connect the memory interface to therest of the memory controller: (a) CLKC—the controller clock 2010; (b)RQ_(C)[i]—the request bus for rank [i] 2020 (typically the same for allranks); (c) TX[j]—the calibration bus for the controller transmit logicslice [j] 2030; (d) RX[j]—the calibration bus for the controller receivelogic slice [i] 2040; (e) Q_(C)[j][3:0]—the read data for slice [j]2050; and (f) D_(C)[j][3:0]—the write data for slice [j] 2060.

Block C1 receives the CLKC signal 2010. Two sets of clock signals arecreated from this reference clock. Here, the clock signals for all ranksare the same, and the clock signals for all slices are the same. Thefirst set of clock signals is for block C2: (a) CLK1 2015—a derivedclock with same frequency as CLKC, and phase-aligned to CLKC; and (b)CLK4[8] 2018—a derived clock with four times the frequency of CLKC.

The second set of clock signals is for block C3: (a) CLK1 2015—a derivedclock with same frequency as CLKC, and phase-aligned to CLKC; (b)CLK4Cyc[1:0] 2022, which is a cycle count of CLK4 clock cycles, and thusindicates a phase of CLK4 cycle relative to CLK1; (c) CLK4CycD[1:0]2025, which is the same as CLK4Cyc[1:0] except that it is delayed byhalf a CLK4 clock cycle relative to CLK4Cyc[1:0]; and (d) CLK4[7:0]2028, which is a set of 8 derived clocks having four times the frequencyof CLKC, each having a different phase offset (as shown in FIG. 22),staggered in increments of ⅛^(th) of a CLK4 cycle. The CLK4[7:0] signalsare also herein called phase vectors and the CLK4Cyc[1:0] is also hereincalled a clock count signal. These phase vectors and the clock countsignal are used by both the transmit and receive circuits for each DQbus.

Block C2 receives the RQ_(C)[i,0] bus 1315 from other circuitry in thecontroller and receives the CLK1 2015 and the CLK4[8] 2018 clock signalfrom block C1.

Block C3 connects to the TX[j], RX[j], Q_(C)[j][3:0], and D_(C)[j][3:0]buses from the rest of the controller. Block C3 receives the CLK1,CLK4Cyc[1:0], CLK4CycD[1:0], and CLK4[7:0] buses from block C1.

FIG. 21 shows the logic 2100 for block C1 of the controller component ofFIG. 20. Block C1 is responsible for creating the derived clock signalsfor blocks C2 and C3 from the reference clock signal CLKC 2010.

The reference clock signal CLKC is received by a PLL circuit 2015, whichproduces a clock signal CLK8 that has eight times the frequency of CLKC.This increase in frequency is set by the circuitry in the feedback loopdescribed below.

The CLK8 signal clocks a three bit register 2118, which produces athree-bit signal asserted on a bus C[2:0] 2019. The three-bit signal onthe C[2:0] bus is decremented by the logic circuit “DEC” 2025 and loadedback into the register 2118 on the next CLK8 edge. Signal C[2] is themost-significant-bit (or “msb”), and signal C[0] 2030 is theleast-significant-bit of the value stored in register 2118. C[2:0]cycles through its values (111, 110, 101, 100, 011, 010, 001, 000, andthen back to 111), with its value being decremented with each cycle ofthe CLK8 signal.

Signal C[2] is buffered by buffer 2058 to produce CLK1 2035. Signal 2035is a derived clock signal that has the same frequency as the referenceclock CLKC. The PLL circuit 2015 compares the CLKC and CLK1 signals onbuses 2010 and 2035, and the phase of the output signal CLK8 is adjusteduntil these two clock signals are essentially phase-aligned (as shown intiming diagram FIG. 22).

Signals C[2] and C[1] are complemented (i.e., inverted) and buffered bybuffers 2045 to produce the CLK4Cyc[1] and CLK4Cyc[0] signals on buses2038 and 2040, respectively. The CLK4Cyc[1] and CLK4Cyc[0] signals areused to label four CLK4 cycles within each CLK1 cycle.

Signals C[2] and C[1] are also loaded into two delay registers 2020clocked by the CLK8 clock signal. The output of these two registers arecomplemented and buffered by buffers 2050 to produce the CLK4CycD[1] andCLK4CycD[0] signals on buses 2052 and 2055 respectively. The CLK4CycD[1]and CLK4CycD[0] signals 2052 and 2055 are the same as the CLK4Cyc[1] andCLK4Cyc[0] signals, delayed by one CLK8 cycle.

Note that all the buffer circuits 2045, 2050, 2058 and capacitive loadsare preferably designed to give the same delay values, so that all theclock signals and clock count signals generated by CLKC (e.g., CLK1,CLK4[8:0], CLK4Cyc[1:0], and CLK4CycD[1:0]) are essentiallyphase-aligned as shown in FIG. 22.

The C[0] signal on bus 2030 has a frequency that is four times that ofthe reference clock signal CLKC 2010. The C[0] signal is the inputsignal to a DLL circuit. There are eight matched delay elements 2060(labeled “D”), each of whose delay is controlled by a “delay-control”signal on line 2065. The delay-control signal could be either a set ofdigital signals, or it could be an analog signal, such as a voltagesignal. The delay of each delay element 2060 is identical.

The output of each delay element 2060 is passed through a buffer 2070(labeled “B”) to produce the nine CLK4[8:0] signals 2075. Here, each ofthese clock signals will have a frequency that is four times that of thereference clock CLKC. The two signals CLK4[0] and CLK4[8] are comparedby the DLL 2080, and the value of delay-control 2065 is adjusted untilthe signals CLK4[0] and CLK4[8] are essentially phase aligned. Theremaining CLK4[7:1] clock signals will have phase offsets that aredistributed in 45° (t_(CLK4Cycle)/8) increments across a CLK4 cycle.

As before, all buffer circuits 2070 and capacitive loads are preferablydesigned to give the same delay values, so that all the CLK4[8:0] clocksignals have evenly distributed phases, and the rising edge of CLK1 willbe essentially aligned to every fourth edge of CLK4[0] and CLK4[8].

FIG. 22 shows the timing diagram with signals 22(a)-(o) for block C1 ofthe memory controller 2000. Block C1 is responsible for creating thederived clock signals for blocks C2 and C3 of system 2000 from thereference clock signal CLKC 2010.

The reference clock signal CLKC is shown in the first waveform 22(a).The cycle time of the CLKC signal is t_(CLK1Cycle). The PLL circuit 2015produces a clock signal CLK8 of 22(b) that here has eight times thefrequency and whose cycle time is t_(CLK8Cycle). The rising edge of theCLK8 signal is delayed from the rising edge of CLKC by t_(PLL), a delayintroduced by the PLL circuit to ensure that the rising edges of CLKCand CLK1 are aligned. The CLK8 signal clocks a three bit register 2118,which produces a bus C[2:0]. This bus decrements through the values{111, 110, 101, 100, 011, 010, 001, 000}, and is delayed from CLK8 byt_(CLK-TO-OUT) (the clock to output delay time of the register 2118).

Signal C[2], is buffered by a buffer 2058 having an associated delay oft_(BUFFER) (arrow 2220) to produce CLK1 2035. Signal 2035, depicted as22(d), is a derived clock signal that has the same frequency as thereference clock CLKC. The PLL circuit 2015 compares the rising edges ofthe two CLKC and CLK1 signals (on buses 2010 and 2035), and the phase ofthe output signal CLK8 is adjusted until these two inputs areessentially phase-aligned. The edges aligned by PLL 2015 are shown byarrows 2210. Note that the following equation will be satisfied when thePLL is phase locked:t _(PLL) +t _(CLK-TO-OUT) +t _(BUFFER) =t _(CLK8Cycle) =t_(CLK1Cycle)/8  (7)

Signals C[2] and C[1] are complemented and buffered to give theCLK4Cyc[1] and CLK4Cyc[0] signals on buses 2038 and 2040, respectively.Signals C[2] and C[1], depicted together as signal 22(e), are alsoloaded into two delay registers 2020 clocked by the CLK8 clock signal.The output of these two registers are complemented and buffered to givethe CLK4CycD[1:0] signals on buses 2052 and 2055, and depicted togetheras signal 22(f). The CLK4CycD[1:0] signals are the same as theCLK4Cyc[1:0] signals delayed by one CLK8 cycle.

The CLK4Cyc[1:0] and CLK4CycD[1:0] signals label the four CLK4 cycleswithin one CLK1 cycle. The two sets of signals are needed because any ofthe eight CLK4[7:0] signals might be used. For example, if a clockdomain is aligned with the CLK4[5:2] clock signals (depicted by theblack dots identified by arrow 2240), then the CLK4Cyc[1] and CLK4Cyc[0]signals are used. If a clock domain is aligned with the CLK4[7,6,0,1]clock signals (represented by arrows 2230), then the CLK4CycD[1] andCLK4CycD[0] signals are used. This alignment with multiple clock domainsgives as much margin as possible for the set and hold times for samplingthe CLK4Cyc[1:0] and CLK4CycD[1:0] signal sets, and permits CLK4 cyclesto be labeled consistently regardless of which CLK4[7:0] signal is used.

The C[0] signal on bus 2030 has a frequency that is four times that ofthe reference clock signal CLKC. The C[0] signal is the input signal tothe DLL circuit containing matched delay elements 2060 and buffers 2070.The output of the eight delay elements 2060 is passed through buffers2070 to produce the CLK4[8:0] signals on bus 2075. Each of these clocksignals has a frequency that is four times that of the reference clockCLKC, as shown by signals 22(g)-(o) in FIG. 22. The two signals CLK4[0]and CLK4[8] are compared by the DLL 2080, and the delay-control value isadjusted until the two signals are essentially phase aligned. The DLLcircuit aligns the edges depicted by arrows 2250. As a result, theCLK4[7:1] clock signals have phase offsets that are distributed in 45°increments (t_(CLK4Cycle)/8) across a CLK4 cycle.

The clock signals CLK4[7:0], signals 22(g)-(n), are used to create theclocks needed for transmitting and receiving in the C3 block of system2000 for each slice of the memory components. Any slice may need any ofthese phase-shifted clock signals. Further, the controller's calibrationcircuitry for a particular slice may select a different clock signalduring system operation, if the timing parameters of the delay pathschange because of temperature and supply voltage variations.

FIG. 23 shows the logic for the controller block 2300 of system 2000.Block 2300, or R0, is part of block C3 (along with block 3000, or T0).Block 2300 is responsible for receiving read data from the memorycomponents and includes three blocks: R1 2400, R2 2500, and R3 2600.

Block R1 connects to the DQ[0,j] bus 1325, which connects to the memorycomponents of the memory system. Block R1 receives the CLKQ[0,j] signal(line 1334) and LoadR[j] signal (line 2310) from block R2. Block R1receives CLK1SkipR[j] (line 2315) and CLK1LevelR[j][1:0] (line 2320)from block R3 and receives CLK1 2015 from outside this controller block2300 (from block C1 in FIG. 20). Block R1 returns read data signalsQ_(C)[j][3:0] to other blocks in the controller.

Block R2 supplies the CLKQ[0,j] and LoadR[j] signals to block R1. BlockR2 receives CLK4BlendR[j][4:0] (line 2325), CLK4PhSelR[j][2:0] (line2330) and CLK4CycleR[j][1:0] (line 2335) from block R3. Block R2 alsoreceives CLK4[7:0], CLK4Cyc[1:0] and CLK4CycD[1:0] from outside of blockR0 (from block C1 in FIG. 20).

Block R3 supplies the CLK4BlendR[j][4:0], CLK4PhSelR[j][2:0] andCLK4CycleR[j][1:0] signals to block R2. Block R3 supplies CLK1SkipR[j]and CLK1LevelR[j][1:0] to block R1. It receives LoadRXA, LoadRXB, CLK1,SelRXB, SelRXAB, IncDecR[j], and 256or1R signals from outside of blockR0 (either block C1 in FIG. 20 or other blocks in the controller).

FIG. 24 shows the logic for the controller block R1 2400 of system 2300.This block is responsible for receiving read data from the memorycomponents and inserting a programmable delay.

The LoadR[j] signal 2310 is asserted on one of every four rising edgesof CLKQ[0,j]. The correct edge is selected in the R1 block. During aread transfer, four one-bit registers 2410 connected serially to theDQ[0,j] bus 1325 continuously shift in the read data that is present onthe DQ[0,j] bus 1325 with each rising edge of CLKQ[0 ,j].

When the LoadR[j] signal is asserted, the most recent shifted-in readdata is loaded in parallel to the 4-bit register 2420. When the Loadsignal is deasserted, the contents of this register are recirculatedthrough the multiplexer 2430 along bus 2435 and held for four CLKQ[0,j]cycles (or one CLK1 cycle).

The Q_(c)[j][3:0] signal on line 2050 and the CLK1 signal on 2015represent two clock domains that may have an arbitrary phase alignmentwith respect to each other, but they will be frequency-locked, here in a4:1 ratio. The serial-to-parallel conversion controlled by LoadR[j] 2310makes the frequencies of the two clock domains identical. Therefore,either the rising edge of CLK1 or the falling edge of CLK1 will becorrectly positioned to sample the parallel data in the four-bitregister 2440. The CLK1SkipR[j] signal (generated in block R3, shown inmore detail in FIG. 26) selects between the two cases. When it is one,the path 2445 with a negative-CLK1-edge-triggered register is enabled,otherwise the parallel register is used directly via path 2448. Ineither case, a positive-CLK1-edge-triggered register samples the outputof the skip multiplexer 2450 and stores the four-bit value in a firstregister 2470.

The final stage involves inserting a delay of zero through three CLK1cycles. This is easily accomplished with a four-to-one multiplexer 2460,and three additional four-bit registers 2470. The CLK1LevelR[j][1:0] bus2320 is generated in block R3, and selects which of the four registers2470 is to be enabled (i.e., it selects which register's output is to bepassed by the multiplexer 2460 onto the Q_(C)[j][3:0] bus 2050).

FIG. 25 shows the logic for the controller block R2 2500. This block isresponsible for creating the CLKQ[0,j] clock signal needed for receivingthe read data from the memory components, and for creating the LoadR[j]signal for performing serial-to-parallel conversion in 2400.

Block R2 supplies the CLKQ[0,j] and LoadR[j] signals to block R1. BlockR2 receives CLK4BlendR[j][4:0] (line 2325), CLK4PhSelR[j][2:0] (line2330) and CLK4CycleR[j][1:0] (line 2335) from block R3. Block R2 alsoreceives CLK4[7:0], CLK4Cyc[1:0] and CLK4CycD[1:0] from outside of blockR0 (from block C1 in FIG. 20).

CLK4PhSelR[j][2:0] on bus 2330 selects which of the eight CLK4[7:0]clock signals will be used as the lower limit for a phase blendingcircuit. The next higher clock signal is automatically selected bymultiplexer 2520 for blending with the lower limit clock signal, whichis selected by multiplexer 2510. For example, if signal 2330 is “010”,then the clock signal used for the lower limit is CLK4[2] and the clocksignal used for the upper limit is CLK4[3]. These are passed by the twoeight-to-one multiplexers 2510 and 2520 to the Phase Blend Logic block2530 via buses 2515 and 2525.

The CLK4BlendR[j][4:0] signal on bus 2325 selects how to interpolatebetween the lower and upper clock signals on buses 2515 and 2525. IfCLK4BlendR[j][4:0] is equal to B, then the interpolated phase is at apoint B/32 of the way between the lower and upper phases. If B is zero,then it is at the lower limit, and if B is 31, then it is almost at theupper limit. The output of the Phase Blend Logic 2530 is CLKQ[0,j], theclock signal on bus 1334 used to sample the read data from the memory.

The Phase Blend Logic 2530 uses well known circuitry, which is thereforenot described in this document. However, the ability to smoothlyinterpolate between two clock signals that have relatively long slewrates (i.e., the rise/fall time of the two signals is greater than thephase difference between the two signals) is important in that it makesthe blending of signals to form a combined signal 1334 andimplementation of dynamic mesochronous systems easier.

The remaining signals and logic in block R2 generate the LoadR[j] signal2310, which indicates when the four read data bits have been seriallyshifted into bit registers 2410 (FIG. 24) and are ready to be clockedinto the parallel register 2420 (FIG. 24). The CLK4CycleR[j][1:0]signal, generated by block R3, picks one of the four possible loadpoints. The LoadR[j] signal on line 2310 is generated by comparingCLK4CycleR[j][1:0] to CLK4Cyc[1:0] using compare logic 2565.CLK4Cyc[1:0] labels the four CLK4 cycles in each CLK1 cycle. However,this comparison must be done carefully, since the LoadR[j] signal 2310is used in the CLKQ[0,j] clock domain, and the CLK4Cyc[1:0] signals aregenerated in the CLK1 domain.

The CLK4CycD[1:0] signals 2540 are delayed from the CLK4Cyc[1:0] signalsby one CLK8 cycle, so there is always a valid bus to use, no matter whatvalue of the CLK4PhSelR[j][2:0] signal is used. The following tablesummarizes the four cases of CLK4PhSelR[j][2:0] that were originallyshown in the timing diagram of FIG. 22:

CLK4Cyc[1:0] CLK4PhSelR[j][2:0] CLK4CycleR[j][1:0] or CLK4CycD[1:0] 00xIncremented CLK4CycD[1:0] 01x not incremented CLK4Cyc[1:0]    10x notincremented CLK4Cyc[1:0]    11x not incremented CLK4CycD[1:0]

The compare logic 2565 generates a positive output (e.g., a “1”) whenits two inputs are equal. The output of the compare logic 2565 issampled by a CLKQ[0,j] register 2575, the output of which is theLoadR[j] signal, and is asserted in one of every four CLKQ[0,j] cycles.

More specifically, AND gate 2580 and multiplexer 2570 determine whethera first input to the compare logic 2565 is CLK4CycleR[j][1:0] or is thatvalue incremented by one by increment circuit 2590. XOR gate 2585 andmultiplexer 2560 determine whether the second input to the compare logic2565 is CLK4Cyc[1:0] or CLK4CycD[1:0], each of which is delayed by oneCLKQ clock cycle by registers 2550 and 2555.

FIG. 26 shows the logic 2600 for the controller block R3 in FIG. 23.Block R3 2600 is responsible for generating the value of clock phasePhaseR[j][1:0] for receiving the read data.

Logic R3 2600 supplies the CLK4BlendR[j][4:0] (line 2325),CLK4PhSelR[j][2:0] (line 2330) and CLK4CycleR[j][1:0] (line 2335) toblock R2. Logic R3 2600 also supplies CLK1SkipR[j] on line 2315 andCLK1LevelR[j][1:0] on line 2320 to block R1 2400. Logic 2600 furtherreceives the LoadRXA 2605, LoadRXB 2610, CLK1, SelRXB 2615, SelRXAB2620, IncDecR[j] 2625, and 256or1R signals 2630 from outside of block2300 (either block C1 or other blocks in the controller).

There are two 12-bit registers (RXA 2635 and RXB 2640) in block 2600.These 12-bit registers digitally store the phase value of CLKQ[0,j] thatwill sample read data at the earliest and latest part of the data windowfor each bit. During normal operation, these two values on registeroutput lines 2637 and 2642 are added by the Add block 2645, and the sumon line 2647 shifted right by one place (to divide by two) by shifter2650, producing a 12 bit value that is the average of the two values(RXA+RXB)/2. Note that the carry-out 2660 of the Add block 2645 is usedas the shift-in of the Shift Right block 2650. In effect, the tworegisters RXA 2635 and RXB 2640 together digitally store a receive phasevalue for a respective slice.

The value (RXA+RXB)/2 is the appropriate value for sampling the readdata with the maximum possible timing margin in both directions. Othermethods of generating an intermediate value are possible. This averagevalue is passed through multiplexer 2670 to become PhaseR[j][11:0] online 2675. The PhaseR[j][4:0], PhaseR[j][7:5] and PhaseR[j][9:8] signalson lines 2676, 2677 and 2678 are extracted from the PhaseR[j][11:0]signal on 2675, and after buffering by buffers 2695 these extractedsignals become the CLK4BlendR[j][4:0], CLK4BlendR[j][7:5], andCLK4BlendR[j][9:8] signals on lines 2325, 2330 and 2335.

The upper two bits of PhaseR[j][11:0] represents the number of CLK1cycles from the t_(OFFSETR) point. The fields CLK1SkipR[j] 2315 andCLK1LevelR[j][1:0] 2320 represent the delay that must be added to thetotal delay of the read data, which is t_(RANGER) no matter what valuePhaseR[j][11:0] contains. Thus, PhaseR[j][11:0] is subtracted from2¹²−2⁸. The factor of “2¹²” represents the maximum value of t_(RANGER).The factor of “2⁸” is needed to give the proper skip value—this will bediscussed further with FIG. 27.

The circuitry adds “111100000000” on line 2680 to the complement ofPhaseR[j][11:0] and asserts carry-in to the adder 2685. The low ninebits of the result are discarded on line 2682, the next bit is bufferedto generate CLK1SkipR[j] and the upper two bits are buffered to generateCLK1LevelR[j][1:0] 2320.

During a calibration operation, the multiplexer 2670 that passes the(RXA+RXB)/2 value instead selects either the RXA register 2635 or theRXB register 2640 directly, as determined by the SelRXAB signal on 2620and the SelRXB signal on 2615. Placing the value in the selectedregister (RXA or RXB) on the PhaseR[j][11:0] bus 2675 causes the receivelogic to set the sampling clock to one side or the other of the datawindow for read data. Once the resulting sampling clock CLKQ[0,j] on1334 (FIG. 25) is stable, the read data is evaluated, and the RXA or RXBvalue is either incremented, decremented, or not changed by logic 2690and output on line 2694. An increment/decrement value of “1” is used forcalibrating the CLKQ[0,j] clock. An increment/decrement value of “256”is used by logic 2690 when the sampling point of the RQ[i,j] bus 1352 inthe memory system component 1310 is changed (because the memory systemcomponent 1310 will change the sampling point of the bus 1352 inincrements of the CLK4 clock cycle). The RQ[i,j] bus sampling point andits calibration process was described above with reference to FIG. 18.

FIG. 27 shows receive timing signals 27(a)-(k) that illustrates fourcases of alignment of the CLKQ[0,j] clock signal 1334 within thet_(RANGER) interval. This diagram illustrates how the following fivebuses are generated: CLK4BlendR[j][4:0] 2325, CLK4PhSelR[j][2:0] 2330,CLK4CycleR[j][1:0] 2335, CLK1SkipR[j] 2315, and CLK1LevelR[j][1:0] 2320.The value of PhaseR[j][11:0] adjusts the values of the signals on thesebuses, and the value of t_(PHASER), which controls the position ofCLKQ[0,j] within the t_(RANGER) interval, and also adjusts thecompensating delays so the overall delay of the read data is(t_(OFFSETR)+t_(RANGER)) regardless of the position of CLKQ[0,j].

The first waveform shows the CLK1 clock signal, 27(a), over t_(RANGER),and the second waveform 27(b) shows the labeling for the four CLK1cycles (i.e., 00, 01, 10, 11) that comprise the t_(RANGER) interval(note there is no bus labeled “CLK1Cyc”; this is shown to make thediagram clearer).

The third waveform, 27(c), shows the CLK4[0] clock signal, and waveform27(d) shows the labeling for the four CLK4 cycles that comprise eachCLK1 cycle.

The fifth waveform, 27(e), shows the numerical values of thePhaseR[j][1:0] bus 2675 as a three digit hexadecimal number. Themost-significant digit includes two bits for the CLK1Cyc value, and twobits for the CLK4Cyc value.

The right side of the diagram 27 shows how three buses are extractedfrom the PhaseR[j][9:0] bus: the CLK4BlendR[j][4:0] 2325,CLK4PhSelR[j][2:0] 2330 and CLK4CycleR[j][1:0] signals 2335 are bufferedversions of the PhaseR[j][4:0] 2676, PhaseR[j][7:5] 2677, andPhaseR[j][9:8] fields 2678, respectively.

The sixth and seventh waveforms, 27(f) and (g), show graphically how theCLK1SkipR[j] and CLK1LevelR[j][1:0] signals on buses 2315 and 2320,respectively, vary as a function of the PhaseR[j][11:0] value. It isnoted that the CLK1SkipR[j] and CLK1LevelR[j][1:0] signals generate acompensating delay for the read data, so they increase from right toleft in FIG. 27.

In FIG. 27(k), or case D, the PhaseR[j][11:0] value is 780₁₆ (2701). Atpoint 2701 of case D, the read data has been sampled and is available ina parallel register (e.g., 2430, FIG. 24) in the CLKQ[0,j] clock domain,and is ready to be transferred to the CLK1 domain. The read data issampled by the next falling edge 2720 of CLK1 1330 at time a00 ₁₆, thenis sampled by the next rising edge 2730 of CLK1 at time c00 ₁₆, andfinally is sampled by the next rising edge 2740 of CLK1 at time 1000 ₁₆.The three intervals labeled “t_(SKIPRN)”, “t_(SKIPR)”, and “t_(LEVELR)”connect the four sampling points 2701-2704. The CLK1SkipR[j] value inwaveform 27(k) is “1” because a “t_(SKIPRN)” interval is used. TheCLK1LevelR[j][1:0] value in waveform 27(k) is “01” because one“t_(LEVELR)” interval is used.

The other cases are analyzed in a similar fashion. In this example, thesize of the t_(RANGER) interval has been chosen to be four CLK1 cycles.It could be easily extended (or shrunk) using the utilizing the methodsthat have been described in this example.

Note that the upper limit of the t_(RANGER) interval is actually 3¾ CLK1cycles because of the method chosen to align the CLK1SkipR[j] andCLK1LevelR[j][1:0] values to the t_(PHASER) values. The loss of the ¼CLK1 cycle of range is not critical, and the method shown gives the bestpossible margin for transferring the read data from the CLKQ[0,j] clockdomain to the CLK1 domain. Other alignment alternatives are possible.The t_(RANGER) could be easily extended by adding more bits toPhaseR[j][11:0] and by adding more Level registers 2470 in FIG. 24.

FIG. 28 shows timing signals 28(a)-(h) that illustrates how the timingvalues are maintained in the RXA and RXB registers 2635 and 2640,respectively. Waveform 28(a) shows the CLK1 signal 1330 in thecontroller 1305. The second waveform, 28(b), shows the RQ[i,0] bus 1315issuing a RDPAT1 command. Signal 28(c) shows the pattern data Q[0,j]returned to the controller. The fourth signal, 28(d), shows the internalclock signal CLKQ[0,j] that samples the data in the controller. Theposition of the CLKQ[0,j] rising edge 2810 is centered on the first bitof pattern data at t_(OFFSETR)+t_(PHASERj)−t_(StoP,Q), where:t _(V,CLK) +t _(PROP,CLKij) +t _(Bij) +t _(SAMPLEij) +t _(CAC,INT) +t_(V,Q) +t _(PROP,Qij) +t _(S,Q) =t _(OFFSETR) +t _(PHASERj) −t_(StoP,Q)  (8)

See FIG. 14A for a graphical representation of this equation. Most ofthe terms on the left side of Eqn. (8) can change as temperature andsupply voltage vary during system operation. The rate of change will berelatively slow, however, so that periodic calibration operations(separated by periods of normal memory operations) can keep thet_(PHASERj) value centered on the read data bits.

As previously discussed, the calibration logic 1355 (see FIGS. 13 and26) maintains two separate register values (RXA and RXB) which track theleft and right side of the read data window 2820. In the lower part ofFIG. 28, the pattern data Q[0,j] and the CLKQ[0,j] rising edge are shownwith an expanded scale. The CLKQ[0,j] rising edge is also shown at threedifferent positions: t_(PHASERj(RXA)), 28(f), t_(PHASERj(Rx)), 28(g),and t_(PHASERj(RXB)), 28(h). The three positions result from setting thePhaseR[j][11:0] signal to the RXA[11:0], RX[11:0] or RXB[11:0] value inlogic 2600, respectively. Here, RX represents the average value of RXAand RXB.

The RXA value shown in 28(f) will hover about the point at which(t_(PHASERj(RXA))−t_(StoP,Q)) trails the start of the Q[0,j][0] bit byt_(S,Q) 2830. If the sampled pattern data is correct (pass), the RXAvalue is decremented, and if the data is incorrect (fail), the RXA valueis incremented.

In a similar fashion, the RXB value shown in 28(h) will hover about thepoint at which (t_(PHASERj(RXB))−t_(StoP,Q)) precedes the end of theQ[0,j][0] bit by t_(H,Q) 2840. If the sampled pattern data is correct(pass), the RXB value is incremented, and if the data is incorrect(fail), the RXA value is decremented.

In both cases, a pass will cause the timing to change in the directionthat makes it harder to pass (reducing the effective set or hold time).A fail will cause the timing to change in the direction that makes iteasier to pass (increasing the effective set or hold time). In thesteady state, the RXA and RXB values will alternate between the twopoints that separate the pass and fail regions. This behavior is alsocalled “dithering”. In a preferred embodiment, calibration of RXA stopswhen the adjustments to RXA change sign (decrement and then increment,or vice versa), and similarly calibration of RXB stops when that valuebegins to dither. Alternatively, the RXA and RXB values can be allowedto dither, since the average RX value will still remain well inside thepass region.

FIG. 29 shows receive timing signals 29(a)-(l) that illustrate acomplete sequence that may be followed for a calibration operation.Waveforms 28(a)-(c) are shown in an expanded view of the pattern readtransaction. The first waveform shows the CLK1 signal in the controller.The second waveform, 29(b), shows the RQ_(C)[i] bus in the controller(see FIG. 20). The third waveform shows the pattern data Q_(C)[j][3:0]in the controller (see FIG. 20). The time interval between the CLK1edges associated with the RDPAT1 command and the returned data P1[3:0]is labeled t_(CAC,C). This value is the same for all slices and allranks in the memory system of the present invention, and is equivalentto (t_(OFFSETR)+t_(RANGER)) or eight CLK1 cycles for this systemexample.

The eight cycle pattern access is one step in the calibration operationshown in waveform 29(d). The calibration sequence for this example takes61 CLK1 cycles (from 02 to 63). Before the sequence begins, all ongoingtransfers to or from memory must be allowed to complete. At thebeginning of the sequence, the SelRXB, SelRXAB, and 256or1R signals29(i), 29(j) and 29(l), respectively, are set to static values that areheld through edge 2920. The following table summarizes the values towhich these signals are set:

Case SelRXB SelRXAB 256or1R RXA calibrate 0 1 0 RXB calibrate 1 1 0

Changing the value of the SelRXAB from 0 to 1 means that a time intervalt_(SETTLE128) (25 CLK1 cycles in this example) 2940 must elapse beforeany pattern commands are issued. This allows the new value ofPhaseR[j][11:0] to settle in the phase selection and phase blendinglogic of the R2 block 2500 (FIG. 25). The pattern data read from thememory component is available in the controller after rising edge 35,shown as edge 2930 in FIG. 29. This pattern data is compared to theexpected value, and a pass or fail determination is made if it matchesor does not match, respectively. The IncDecR[j] signal 2625 is assertedor deasserted, as a result, and the LoadRXA 2605 or LoadRXB 2610 signalis pulsed for one CLK1 cycle to save the incremented or decrementedvalue, as shown in waveforms 29(g), 29(h) and 29(k). The following tablesummarizes the values to which these signals are set:

Case IncDecR[j][1:0] LoadRXA LoadRXB RXA calibrate (pass) 11 1 (pulse) 0RXA calibrate (fail) 01 1 (pulse) 0 RXB calibrate (pass) 01 0 1 (pulse)RXB calibrate (fail) 11 0 1 (pulse)

At rising edge 38 (2920), all signals 29(g)-(l) can be returned to zero.Changing the value of the SelRXAB from 1 to 0 means that another timeinterval t_(SETTLE128) 2950 must elapse before any read or writecommands 2960 are issued.

Note that the calibration sequence may be performed on all slices of thememory system in parallel. All of the control signals can be sharedbetween the slices except for IncDecR[j], which depends upon thepass/fail results for the pattern data for that slice.

In preferred embodiments, the calibration sequence is performed for RXAand RXB at periodic intervals that are spaced closely enough to ensurethat timing adjustments can keep up with timing changes due to, forexample, temperature and supply voltage variations.

When the sampling point of the RQ[i,j] bus 1352 by the CLKB[i,j] clocksignal 1347 is changed (as in FIG. 18), the sampling point of theCLKQ[0,j] receive clock 1334 in the controller must be adjusted. This isaccomplished by an update sequence for the RXA and RXB register values.This update sequence is similar to the calibration sequence of FIG. 29,but with some simplifications. Preferably, this update sequence isperformed immediately after the RQ sampling point was updated.

When updating the RXA and RXB registers to compensate for a change inthe sampling point of the RQ[i,j] bus, the SelRXB, SelRXAB, and 256or1Rsignals are set to static values that are held through rising edge 38(edge 2920). The PhaseR[j][11:0] is not changed (SelRXAB remains low),so that the pattern transfer does not need to wait for circuitry tosettle as in the calibration sequence. The following table summarizesthe values to which these signals are set:

Case SelRXB SelRXAB 256or1R RXA update 0 0 1 RXB update 1 0 1

The reason that an increment/decrement value of 256 is used instead of 1is because when the sample point of the RQ[i,j] bus is changed, it willbe by {+1,0,−1} CLK4 cycles. A CLK4 cycle corresponds to the value of256 in the range of PhaseR[j][11:0].

When the sample point changes by a CLK4 cycle, the data that is receivedin the Q[j][3:0] bus 2050 will shift by one bit to the right or left. Bycomparing the retrieved pattern data to the expected data, it can bedetermined whether the RXA and RXB values need to be increased ordecreased by 256, or left the same. The following table summarizes thevalues to which these signals are set:

Case IncDecR[j][1:0] LoadRXA LoadRXB RXA update (shifted right) 11 1(pulse) 0      RXA update (pass) 00 1 (pulse) 0      RXA update (shiftedleft) 01 1 (pulse) 0      RXB update (shifted right) 11 0      1 (pulse)RXB update (pass) 00 0      1 (pulse) RXB update (shifted left) 01 0     1 (pulse)

Both RXA and RXB can be updated successively using the same pattern readtransfer. Note that the update sequence may be performed on all slicesin parallel. All of the control signals can be shared between the slicesexcept for IncDecR[j], which depends upon the pass/fail results for thepattern data for that slice.

Once the update sequence has completed, a time interval t_(SETTLE256)(e.g., 50 CLK1 cycles) must elapse before any read commands are issued.This allows the new value of PhaseR[j][11:0] to settle in the phaseselection and phase blending logic of the R2 block (FIG. 25).

Before the RXA and RXB register values can go through the calibrationand update sequences just described, they must be initialized toappropriate starting values. This can be done relatively easily with thecircuitry that is already in place.

The initialization sequence begins by setting the RXA register to theminimum value of 000₁₆ and by setting the RXB register to the maximumvalue fff₁₆. These will both be failing values, but when the calibrationsequence is applied to them, both values will move in the properdirection (RXA will increment and RXB will decrement).

Thus, the initialization procedure involves performing the RXAcalibration repeatedly until it passes. Then the RXB calibration will beperformed repeatedly until it passes. The settings of the varioussignals will be:

Case SelRXB SelRXAB 256or1R RXA calibrate  0 1      0      RXB calibrate 1 1      0      Case IncDecR[j][1:0] LoadRXA LoadRXB RXA calibrate(pass) 11 1 (pulse) 0      RXA calibrate (fail) 01 1 (pulse) 0      RXBcalibrate (pass) 01 0      1 (pulse) RXB calibrate (fail) 11 0      1(pulse)

There will be approximately 3840 (=4096−256) iterations performed duringthe initial calibration, since the total range is 4096 and 256 is themaximum width of a bit.

Each iteration can be done with little settling time, t_(SETTLE1),because the RXA or RXB value will change by only a least-significant-bit(and therefore the t_(SETTLE1) time will be very small). It will stillbe necessary to observe a settling time at the beginning and end of eachiteration sequence in which the PhaseR[j][11:0] value is changed bylarge amounts. PhaseR[j][11:0] changes by large amounts when SelRXAB ischanged in the normal calibration process described earlier.

Note that the initialization sequence may be performed on all slices inparallel. All of the control signals can be shared between the slicesexcept for IncDecR[j], which depends upon the pass/fail results for thepattern data for that slice.

FIG. 30 shows the logic 2000 for the controller block T0. Block T0 ispart of block C3 of FIG. 20 (along with block 2300). Block T0 isresponsible for transmitting the write data to the memory component. Itconsists of three blocks: T1 3100, T2 3200, and T3 3300.

Block T1 connects to bus DQ[0,j] 1325, which connects to the externalmemory system (see FIG. 13). Block T1 receives the CLKD[0,j] 1332 andLoadT[j] 3010 signals from block T2 and receives CLK1SkipT[j] 3015 andCLK1LevelT[j][1:0] 3020 signals from block T3. Block T1 also receivesCLK1 1330 from outside of block T0 (e.g., from block C1 in FIG. 20).Block T1 also returns D_(C)[j][3:0] to other blocks in the controller.

Block T2 supplies the CLKD[0,j] 1332 and LoadT[j] 3010 signals to blockT1. Block T2 receives CLK4BlendT[j][4:0] 3025, CLK4PhSelT[j][2:0] 3030and CLK4CycleT[j][1:0] 3035 from block T3. Block T2 receives CLK4[7:0]2075, CLK4Cyc[1:0] 2038, 2040 and CLK4CycD[1:0] 2052, 2055 from block C1of the controller (see FIG. 20).

Block T3 supplies the CLK4BlendT[j][4:0], CLK4PhSelT[j][2:0] andCLK4CycleT[j][1:0] signals to block T2. Block T3 also suppliesCLK1SkipT[j] 3015 and CLK1LevelT[j][1:0] 3020 to block T1. Block T3 alsoreceives LoadTXA, LoadTXB, CLK1, SelTXB, SelTXAB, IncDecT[j], and256or1T signals (3040-3065) from outside of block T0 (either from blockC1 or from other blocks in the controller, via the TX[j] control bus2030).

FIG. 31 is a logic diagram of controller block T1 3100, which isresponsible for transmitting write data on bus 2060 from memory andinserting a programmable delay before transmitting onto the DQ[0,j] bus.

Block T1 connects to the DQ[0,j] bus 1325, which connects to an externalmemory system. Block T1 receives the CLKD[0,j] and LoadT[j] signals fromblock T2. It receives CLK1SkipT[j] 3015 and CLK1LevelT[j][1:0] signalsfrom block T3. Block T1 receives CLK1 from outside of block T0 (e.g.,from block C1) and receives D_(C)[j][3:0] from other blocks in thecontroller.

The first stage of the T1 Block inserts a delay of zero through threeCLK1 cycles. The data received from the D_(C)[j][3:0] bus 2060 isinitially stored in a four-bit register 3105. Delay insertion isaccomplished using a four-to-one multiplexer 3110, and three additionalfour-bit registers 3115. The CLK1LevelT[j][1:0] bus 3020 can begenerated in block T3 from bus 3020, and selects the data from one ofthe four registers 3105, 3115 for the multiplexer 3110 to pass.

The CLKD[0,j] and CLK1 clock signals may have an arbitrary phasealignment, but they will be frequency-locked in a 4:1 ratio. Either therising edge of CLK1 or the falling edge of CLK1 can be positioned todrive the parallel data into the four-bit register 3120 clocked byCLKD[0,j]. The CLK1SkipT[j] signal on line 3015 (generated in block T3)selects between the two cases through a skip multiplexer 3150. When itis one, the path 3165 with a negative-CLK1-edge-triggered register isenabled, otherwise the direct path 3160 to multiplexer 3150 is used. Ineither case, a positive-CLKD[0,j]-edge-triggered register 3120 samplesthe output 3170 of the skip multiplexer 3150.

When the LoadT[j] signal 3010 is asserted, the most recently loaded4-bit value in register 3120 is loaded into the four one-bit registers3130 connected serially to the DQ[0,j] bus 1325. When the Load signal isdeasserted, the contents of the four one-bit registers 3130 are shiftedserially to the DQ[0,j] bus through multiplexer 3140.

FIG. 32 shows the logic for the controller block T2 3200, which isresponsible for creating the CLKD[0,j] clock on line 1332 needed fortransmitting the write data to the memory component 1310 as shown inFIG. 15.

Block T2 supplies the CLKD[0,j] and LoadT[j] signals to block T1 of FIG.31. Block T2 receives CLK4BlendT[j][4:0], CLK4PhSelT[j][2:0] andCLK4CycleT[j][1:0] from block T3. Block T2 receives CLK4[7:0],CLK4Cyc[1:0] and CLK4CycD[1:0] from outside of block T0 (from block C1).

The CLK4PhSelT[j][2:0] signal on line 3030 selects which of the eightCLK4[7:0] clock signals will be selected by multiplexer 3220 as the“lower limit clock signal” for the phase blending logic 3210. TheCLK4PhSelT[j][2:0] signal is also used by multiplexer 3215 to select thenext higher clock signal for blending. For example, ifCLK4PhSelT[j][2:0] is “010”, then the clock signal used for the lowerlimit is CLK4[2] and the clock signal used for the upper limit isCLK4[3]. These limit signals are passed by the two eight-to-onemultiplexers 3215, 3220 to the Phase Blend Logic block 3210 on lines3222 and 3224, respectively.

The CLK4BlendT[j][4:0] signal on bus 3025 selects how to interpolatebetween the lower and upper clock signals in phase blend logic 3210. Forexample, if CLK4BlendT[j][4:0] is equal to B, then the interpolatedphase is at a point B/32 of the way between the lower and upper phases.If B is zero, then it is at the lower limit set by multiplexer 3220, andif B is 31 set by multiplexer 3215, then it is almost at the upperlimit. The output of the Phase Blend logic 3210 is the CLKD[0,j] clocksignal on bus 1332, used to write data to a memory component.

The Phase Blend Logic 3210 uses well known circuit techniques tosmoothly interpolate between two clock signals, and thus is notdescribed in detail in this document.

The remaining signals and logic of the T2 block generate the LoadT[j]signal 3010, which indicates when the four write data bits are to beshifted into the serial registers 3130. The CLK4CycleT[j][1:0] signal,generated by the T3 block, picks one of the four possible load pointsfor multiplexer 3140. The LoadT[j] signal is generated by using comparelogic 3265 to compare CLK4CycleT[j][1:0] to CLK4Cyc[1:0], which labelsthe four CLK4 cycles in each CLK1 cycle. However, this comparison mustbe done carefully, since the LoadT[j] signal 3010 is used in theCLKD[0,j] clock domain, and the CLK4Cyc[1:0] signals 2038, 2040 aregenerated in the CLK1 domain.

The CLK4CycD[1:0] signals on lines 2052, 2055 are delayed from theCLK4Cyc[1:0] signals by one CLK8 cycle, so there is always a valid busto use, no matter what value of CLK4PhSelT[j][2:0] is used. See thetiming diagram of FIG. 22. The following table summarizes the four casesof CLK4PhSelT[j][2:0]:

CLK4Cyc[1:0] CLK4PhSelT[j][2:0] CLK4CycleT[j][1:0] or CLK4CycD[1:0] 00xincremented CLK4CycD[1:0] 01x not incremented CLK4Cyc[1:0]    10x notincremented CLK4Cyc[1:0]    11x not incremented CLK4CycD[1:0]

The output of the compare logic 3265 is sampled by a CLKD[0,j] register3275 to generate the LoadT[j] signal. The LoadT[j] signal is asserted inone of every four CLKD[0,j] cycles.

More specifically, AND gate 3280 and multiplexer 3270 determine whethera first input to the compare logic 3265 is CLK4CycleT[j][1:0] or is thatvalue incremented by one by increment circuit 3290. XOR gate 3285 andmultiplexer 3260 determine whether the second input to the compare logic3265 is CLK4Cyc[1:0] or CLK4CycD[1:0], each of which is delayed by oneCLKQ clock cycle by registers 3250 and 3255.

FIG. 33 corresponds to FIG. 26 and shows the logic 3300 for thecontroller block T3, which is part of block T0. The T3 block isresponsible for generating the value of clock phase CLKD[0,j] fortransmitting write data.

Block T3 supplies the CLK4BlendT[j][4:0], CLK4PhSelT[j][2:0] andCLK4CycleT[j][1:0] signals to block T2. Block T3 supplies CLK1SkipT[j]and CLK1LevelT[j][1:0] to block T1. Block T3 receives LoadTXA, LoadTXB,CLK1, SelTXB, SelTXAB, IncDecT[j], and 256or1T signals from outside ofblock T0 (either block C1 of FIG. 20 or other blocks in the controller).

As with block R3, there are two 12-bit registers here (TXA 3335 and TXB3340) in block T3. These registers digitally store the phase value ofCLKD[0,j] that will transmit write data at the earliest and latest partof the data window for each bit. During normal operation, these twovalues on lines 3337 and 3342 are added by the Add block 3345, andshifted right by one place by shifter 3350 (to divide by two), producinga 12 bit value on line 3355 that is the average of the two values(TXA+TXB)/2. Note that the carry-out of the Add block 3345 is used asthe shift-in of the shift right block 3350. In effect, the two registersTXA 3335 and TXB 3340 together digitally store a transmit phase valuefor a respective slice.

The average value (TXA+TXB)/2 is the appropriate value for transmittingthe write data with the maximum possible timing margin in bothdirections. Other methods of generating an intermediate value arepossible. This average value is passed through multiplexer 3370 tobecome PhaseT[j][11:0] on bus 3375. The CLK4BlendT[j][4:0],CLK4PhSelT[j][2:0] and CLK4CycleT[j][1:0] signals (on buses 3025, 3030,3035) are generated by extracting PhaseT[j][4:0] 3376, PhaseT[j][7:5]3377, and PhaseT[j][9:8] 3378 fields, respectively from the PhaseT[j][11:0] signal and buffering the extracted signals with buffers 3395.

The upper bits of PhaseT[j][11:0] 3375 represent the number of CLK1cycles from the t_(OFFSETT) point. The fields CLK1SkipT[j] 3015 andCLK1LevelT[j][1:0] 3020 are extracted from these upper bits. Thus,PhaseT[j][11:0] is added to −2⁸. The factor of “2⁸” is needed to givethe proper skip value (this will be discussed further with FIG. 34).

An adder 3385 adds “111100000000” on line 3380 to PhaseT[j][11:0]. Thelowest nine bits of the result are discarded on line 3382, the next bitis buffered to produce CLK1SkipT[j] 3015 and the upper two bits arebuffered to produce CLK1LevelT[j][1:0] 3020. Note that here theCLK1SkipT[j] and CLK1LevelT[j][1:0] fields come from addingPhaseT[j][11:0] to a constant, whereas for the R3 block (FIG. 26) of thecontroller's receive calibration circuitry, the CLK1SkipR[j] andCLK1LevelR[j][1:0] fields come from subtracting PhaseR[j][11:0] from aconstant.

During a calibration operation, the multiplexer 3370 used to select theTX value ((TXA+TXB)/2) instead selects either the TXA register 3335 orthe TXB register 3340. Placing this value on the PhaseT[j][11:0] bus3375 causes the transmit logic to set the driving clock to one side orthe other of the data window for write data. Once the driving clockCLKD[0,j] on line 1332 is stable, data is written to a memory componentand read back and evaluated, and the TXA or TXB value is eitherincremented, decremented, or not changed by logic 3390. Anincrement/decrement value of “1” is used for calibrating the CLKD[0,j]clock. The increment/decrement value of “256” is used by logic 3390 whenthe sampling point of the RQ[i,j] bus in memory is changed (the memorycomponent will change the sampling point of the RQ[i,j] bus 1352 inincrements of the CLK4 clock cycle). The RQ[i,j] bus sampling point andits calibration process are described above with reference to FIG. 18.

FIG. 34 shows transmit timing signals 34(a)-(k) that illustrate fourcases of alignment of the CLKD[0,j] clock signal 1334 within thet_(RANGET) interval 3405. This diagram illustrates how the signals onthe following five buses are generated: CLK4BlendT[j][4:0] 3025,CLK4PhSelT[j][2:0] 3030, CLK4CycleT[j][1:0] 3035, CLK1SkipT[j] 3015, andCLK1LevelT[j][1:0] 3020. The value of PhaseT[j][11:0]adjusts the valueof these buses and the value of t_(PHASET) (the position of CLKD[0,j]within the t_(RANGET) interval).

The first waveform 34(a) shows the CLK1 clock signal over an interval ofduration equal to t_(RANGET), and the second waveform 34(b) shows thelabeling for the four CLK1 cycles that comprise the t_(RANGET) interval(note there is no bus labeled “CLK1Cyc”; this is shown to make thediagram clearer).

The third waveform 34(c) shows the CLK4[0] clock signal, and waveform34(d) shows the labeling for the four CLK4 cycles that comprise eachCLK1 cycle (note—a CLK4Cyc[1:0] bus does exist).

The fifth waveform 34(e) shows the numerical values of thePhaseT[j][11:0] bus 3375 as a three digit hexadecimal number. Themost-significant digit includes two bits for the CLK1Cyc value, and twobits for the CLK4Cyc value.

The right side of FIG. 34 shows how three buses are extracted from thePhaseT[j][9:0] bus: the CLK4BlendT[j][4:0] 3025, CLK4PhSelT[j][2:0] 3030and CLK4CycleT[j][1:0] 3035 signals are generated from thePhaseT[j][4:0] 3376, PhaseT[j][7:5] 3377, and PhaseT[j][9:8] 3378 fieldsof the PhaseT[j][11:0] signal, respectively.

The sixth and seventh waveforms, 34(f) and 34(g), show graphically howthe CLK1SkipT[j] and CLK1LevelT[j][1:0] buses 3015 and 3020,respectively, vary as a function of the PhaseT[j][11:0] value. Thesebuses increase in value from left to right in the figure. Note that thisis opposite from the direction for the receive case.

In case A, shown at 34(h), the PhaseT[j][11:0] value is 880₁₆ (3410).The write data is sampled 3401 on the rising edge 3415 of CLK1 at time000 ₁₆ 3420. The data is sampled 3402 at 400 ₁₆ (3430) by the nextrising edge 3435 of CLK1. The associated CLK1LevelT[j][1:0] in FIG.34(g) value is “01” because one “t_(LEVELT)” interval is used. The writedata is sampled 3403 by the next falling edge 3440 of CLK1 at time60016. The CLK1SkipT[j] value is “1” at “time” 880 ₁₆ because a“t_(SKIPTN)” interval 3445 is used. The write data then crosses into theCLKD[0,j] clock domain, and is sampled 3404 by the rising edge ofCLKD[0,j]. The three intervals labeled “t_(LEVELT)”, “t_(SKIPTN)”, and“t_(SKIPT)” connect the four sampling points 3401-3404.

The other cases B-D shown at 34(i)-(k) are analyzed in a similarfashion. In this example, the size of the t_(RANGET) 3405 interval hasbeen chosen to be four CLK1 cycles. The interval could be easilyextended (or reduced) using the utilizing the methods that have beendescribed in this example.

Note that the upper limit of the t_(RANGET) interval is actually 3¾ CLK1cycles because of the method chosen to align the CLK1SkipT[j] andCLK1LevelT[j][1:0] values to the t_(PHASET) values (the first ¼ CLK1cycle cannot be used). The loss of the ¼ CLK1 cycle of range is notcritical, and the method shown gives the best possible margin fortransferring the write data to the CLKD[0,j] clock domain from the CLK1domain. Other alignment methods are possible. The range of t_(RANGET)values could be easily extended by adding more bits to thePhaseT[j][11:0] value and by adding more level registers 3115 in FIG.31.

FIG. 35 shows timing signals 35(a)-(j) that illustrate how timing valuesare maintained in the TXA and TXB registers 3335 and 3340, respectively.Waveform 35(a) shows the CLK1 signal in the controller 1305. The secondwaveform, 35(b), shows the RQ[i,0] bus 1315 issuing a WRPAT0 (write toPAT0 register) command 3510. Waveform 35(c) shows the pattern dataD[i,j] received at memory component [i,j]. The fourth waveform 35(d)shows the internal clock signal CLKD[0,j] that drives the data from thecontroller. The position of the CLKD[0,j] rising edge is centered ont_(OFFSETT)+t_(PHASETj), where:t _(V,CLK) +t _(PROP,CLKij) +t _(Bij) +t _(SAMPLEij) +t _(CWD,INT) −t_(StoP,D) −t _(S,D) −t _(PROP,Dij) −t _(V,D) =t _(OFFSETT) +t_(PHASETj)  (9)

Most of the terms on the left side of Eqn. (9) can change as temperatureand supply voltage vary during system operation. The rate of change willbe relatively slow, however, so that periodic calibration operations cankeep the t_(PHASETj) value centered on the write data bits.

The calibration is accomplished by maintaining two separate registervalues (TXA and TXB).which track the left and right side of the writedata window 3520, respectively, shown in the lower part of FIG. 35. Thepattern data D[i,j] and the CLKD[0,j] rising edge are shown with anexpanded scale in waveforms 35(e)-(j). The CLKD[0,j] rising edge is alsoshown at three different positions: t_(PHASETj(TXA)), t_(PHASETj(TX))and t_(PHASETj(TXB)) in waveforms 35(f), 35(h) and 35(j), respectively.These three positions result from setting the PhaseT[j][11:0] bus 3375to carry the TXA[11:0], TX[11:0], and TXB[11:0] signals in logic 3300,respectively. Here, TX represents the average value of TXA and TXB.

The TXA value, shown at 35(f), will hover about the point 3525 at whicht_(PHASETj(TXA)) precedes the end of the D[i,j][0] bit byt_(V,D,MIN)+t_(CLK4CYCLE) (3530). If the sampled pattern data is correct(pass), the TXA value is decremented, and if the data is incorrect(fail), the TXA value is incremented. Note that the D[i,j][0] bit issampled by the memory component, which requires a data window of t_(S,D)3540 and t_(H,D) 3550 on either side of the sampling point 3545. Alsonote that the sampled write data must be returned to the controller toevaluate the pass/fail criterion. However, this return of sampled writedata might not be required in other implementations.

In a similar fashion, the TXB value shown in FIG. 350) will hover aboutthe point 3555 at which t_(PHASETj(TXB)) precedes the start of theD[i,j][0] bit by t_(V,D,MAX) (3560). If the sampled pattern data iscorrect (pass), the TXB value is incremented, and if the data isincorrect (fail), the TXA value is decremented.

For both TXA and TXB values, a pass will cause the timing to change inthe direction that makes it harder to pass (reducing the effective datawindow size). A fail will cause the timing to change in the directionthat makes it easier to pass (increasing the effective data windowsize). In the steady state, the TXA and TXB values will alternatebetween the two points that separate the pass and fail regions. As notedearlier, this behavior is called “dithering”. In a preferred embodiment,calibration of TXA stops when the adjustments to TXA change sign(decrement and then increment, or vice versa), and similarly calibrationof TXB stops when that value begins to dither. Alternatively, the TXAand TXB values can be allowed to dither, since the average TX value willstill remain well inside the pass region.

FIG. 36 shows transmit timing signals 36(a)-(m) that illustrate thecomplete sequence that is followed for a calibration timing operationfor TXA/TXB. Waveforms 36(a)-(d) are shown in an expanded view of thepattern read transaction. The first waveform 36(a) shows the CLK1 signaland the second waveform 36(b) shows the RQ_(C) bus in the controller(see FIG. 20). The third waveform 36(c) shows the pattern dataD_(C)[j][3:0] in the controller (see FIG. 20). The fourth timingwaveform 36(d) shows the returned pattern data Q_(C)[j][3:0] in thecontroller. Note that the time interval between the CLK1 edgesassociated with the WRPAT0 command 3605 and RDPAT0 command 3610 ist_(WR,RD) 3615. Parameter 3615 is two CLK1 cycles in this example. Alsonote that the time interval between the CLK1 edges associated with theRDPAT0 command 3610 and the returned data P0[3:0] 3620 is t_(CAC,C)3625, the same as for the receive calibration and normal read operationsdescribed relative to FIG. 29.

The ten cycle pattern access is one step in the calibration operationshown in the lower part of the diagram in timing signals 36(e)-(m). Thecalibration sequence for this example takes 63 CLK1 cycles (here,associated with 00 to 63 in 36(e)). Before the sequence begins, allongoing transfers to or from the memory components must be allowed tocomplete. At the beginning of the sequence, the SelTXB, SelTXAB, and256or1T signals (see 3050, 3055, 3065 in FIG. 30) are set to staticvalues that are held through rising edge 38, associated with time 3630.The following table summarizes the values to which these signals areset:

Case SelTXB SelTXAB 256or1T TXA calibrate 0 1 0 TXB calibrate 1 1 0

Changing the value of the SelTXAB from 0 to 1 means that a time intervalt_(SETTLE128) (25 CLK1 cycles in this example) must elapse before anypattern commands are issued. The time elapse allows the new value ofPhaseT[j][11:0] 3375 to settle in the phase selection and phase blendinglogic 3210 of the T2 block. The pattern data 3640 is available in thecontroller after rising edge 35 associated with time 3650. The new valueis compared to the expected value, and a pass or fail determination ismade if it matches or does not match, respectively. The IncDecT[j]signal in block T3 (FIG. 30) is asserted or deasserted, as a result, andthe LoadTXA or LoadTXB signal (signal 36(h)) is pulsed for one CLK1cycle to save the incremented or decremented value. The following tablesummarizes the values to which these signals are set:

Case IncDecT[j][1:0] LoadTXA LoadTXB TXA calibrate (pass) 11 1 (pulse) 0TXA calibrate (fail) 01 1 (pulse) 0 TXB calibrate (pass) 01 0 1 (pulse)TXB calibrate (fail) 11 0 1 (pulse)

At rising edge 38 (at time 3630), all signals can be returned to zero.Changing the value of the SelTXAB from 1 to 0 as shown in signal 36(j)means that another time interval t_(SETTLE128) must elapse before anyread or write commands 3655 are issued. The calibration sequencedescribed may be performed on all slices of a memory system in parallel.Control signals can be shared between the slices except for IncDecT[j],which depends upon the pass/fail results for the pattern data for thatslice.

The calibration sequence must be performed for registers TXA 3335 andTXB 3340 at periodic intervals that are spaced closely enough to ensurethat timing adjustments can keep up with timing changes due to, forexample, temperature and supply voltage variations.

When the sampling point of the RQ[i,j] bus 1352 by the CLKB[i,j] clocksignal on line 1347 (FIG. 13) is changed in the memory component, thedriving point of the CLKD[0,j] transmit clock on line 1332 in thecontroller must be adjusted. The adjustment is accomplished by an updatesequence for the TXA and TXB register values. This is similar to thecalibration sequence of FIG. 36, but with some simplifications.Typically this update sequence would be performed immediately after theRQ sampling point was updated.

The SelTXB 3050, SelTXAB 3055, and 256or1T 3065 signals are set tostatic values that are held through the update sequence. ThePhaseT[j][11:0] value 3375 is not changed (SelTXAB remains low), so thatthe pattern transfer doesn't need to wait for circuitry to settle as inthe calibration sequence. The following table summarizes the values towhich these signals are set:

Case SelTXB SelTXAB 256or1T TXA update 0 0 1 TXB update 1 0 1

The reason that an increment/decrement value of 256 is used instead of 1is because when the sample point of the RQ[i,j] bus is changed, it willbe by {+1,0, −1} CLK4 cycles. A CLK4 cycle corresponds to the value of256 in the range of PhaseT[j][11:0]. When the sample point changes by aCLK4 cycle, the data that is received in the D[j][3:0] bus 2060 willshift by one bit to the right or left. By comparing the pattern data tothe expected data, it can be determined whether the TXA and TXB valuesneed to be increased or decreased by 256, or left the same. Thefollowing table summarizes the values to which these signals are set:

Case IncDecT[j][1:0] LoadTXA LoadTXB TXA update (shifted right) 11 1(pulse) 0 TXA update (pass) 00 1 (pulse) 0 TXA update (shifted left) 011 (pulse) 0 TXB update (shifted right) 11 0 1 (pulse) TXB update (pass)00 0 1 (pulse) TXB update (shifted left) 01 0 1 (pulse)

Both TXA and TXB can be updated successively using the same pattern readtransfer. Note that the update sequence may be performed on all slicesin parallel. The control signals can be shared between the slices exceptfor IncDecT[j], which depends upon the pass/fail results for the patterndata for that slice.

Once the update sequence has completed, a time interval t_(SETTLE256)must elapse before any write commands are issued. This time elapseallows the new value of PhaseT[j][11:0] to settle in the phase selectionand phase blending logic 3210 of the T2 block (FIG. 32).

Before TXA and TXB register values can go through the calibration andupdate sequences just described, the values must be initialized toappropriate starting values. This initialization can be done relativelyeasily with the circuitry that is already in place.

The initialization sequence begins by setting the TXA register 3335 tothe minimum value of 000₁₆ and by setting the TXB register 3340 to themaximum value fff₁₆. These will both be failing values, but when thecalibration sequence is applied to them, both values will move in theproper direction (TXA in increment and TXB will decrement).

Thus, the initialization procedure involves performing the TXAcalibration repeatedly until it passes. Then the TXB calibration will beperformed repeatedly until it passes. The settings of the varioussignals will be:

Case SelTXB SelTXAB 256or1T TXA calibrate  0 1      0      TXB calibrate 1 1      0      Case IncDecT[j][1:0] LoadTXA LoadTXB TXA calibrate(pass) 11 1 (pulse) 0      TXA calibrate (fail) 01 1 (pulse) 0      TXBcalibrate (pass) 01 0      1 (pulse) TXB calibrate (fail) 11 0      1(pulse)

There will be approximately 3840 (=4096−256) iterations performed (sincethe total range is 4096 and 256 is the maximum width of a bit).

Each iteration can be done with little settling time, t_(SETTLE1),because the TXA or TXB value will change by only a least-significant-bit(and therefore the t_(SETTLE1) time will be very small). It will stillbe necessary to observe a settling time at the beginning and end of eachiteration sequence when the PhaseT[j][11:0] value is changed by a largeamount. PhaseT[j][11:0] changes by large amounts when SelTXAB is changedin the normal calibration process described above.

Note that the initialization sequence may be performed on all slices inparallel. All of the control signals can be shared between the slicesexcept for IncDecT[j], which depends upon the pass/fail results for thepattern data for that slice.

Calibration State Machine Logic

FIG. 37 shows a sample block diagram 3900 of the logic for performingthe calibration processes that have been described above. For example,these processes include the calibration process in which thePhaseT[j][1:0] or PhaseR[j][1:0] values are incremented or decrementedby one, the “update” calibration process in which the PhaseT[j][11:0] orPhaseR[j][11:0] values are incremented or decremented by 256, the“initialize” calibration process in which the PhaseT[j][11:0] orPhaseR[j][11:0] values are incremented or decremented from an initialvalue, by up to 4096 bits, until an initial calibration is achieved, andthe “CALSET/CALTRIG” calibration process in which the RQ sampling cycleof the memory components is updated. FIGS. 18, 29 and 36 are timingdiagrams illustrating the pulsing of the control signals that performthe steps of each calibration process. The logic in FIG. 37 drives thesecontrol and data signals. The control and data signals include two setsof signals that are carried on busses that connect to the C3 block inFIG. 20. The first set of control signals includes signals 2060(D_(C)[j][3:0]), 3910 (LoadTXA, LoadTXB, SelTXB, SelTXAB and 256or1T)and 3915 (IncDecT[j][1:0]). This first set of signals corresponds tosignals 2030 and 2060 in FIG. 20. The second set of control and datasignals includes signals 2050 (Q_(C)[j][3:0]), 3925 (LoadRXA, LoadRXB,SelRXB, SelRXAB and 256or1R) and 3930 (IncDecR[j][1:0]). This second setof signals corresponds to signals 2040 and 2050 in FIG. 20.

The control signals also include the set of signals 3920(RQ_(C)[i][N_(RQ)-1:0]) that connect to the C2 block in FIG. 20.

The logic 3900 is controlled by the following signals:

CLKC 2010, the primary clock used by the memory controller;

CalStart 3945, a signal that is pulsed to indicate that a calibrationoperation should be performed;

CalDone 3950, a signal that is pulsed to indicate the calibrationoperation is completed; and

CalType 3955, a five bit bus that specifies which calibration operationis to be performed.

The controller is configured to ensure that the RQ_(C)[i][N_(RQ)-1:01]3920, Q_(C)[j][3:0] 2050, or D_(C)[j][3:0] 2060 buses are not busy withnormal read or write operations when the calibration operation isstarted. This is preferably accomplished with a hold-off signal (notshown) that is sent to the controller and which allows presentlyexecuting read or write operations to complete, and prevents any queuedread or write operations from starting. Additionally, a decision tostart a calibration operation could be made by a timer circuit (notshown), which uses a counter to measure the time interval betweensuccessive calibration operations. It is also possible that acalibration operation could be started early if there was an idleinterval that allowed it to be performed with less interference with thenormal read and write transactions. In any case, the calibrationoperations can be scheduled and executed with only hardware. However,this hardware-only characteristic would not preclude using a softwareprocess in some systems to either schedule calibration operations, or toperform the sequence of pulsing on the control signals. It is likelythat a full hardware implementation of the calibration logic would bepreferred for use in most systems because of the ease of design andconvenience of operation.

In a preferred embodiment, the CalType[2:0] signal selects the type ofcalibration operation using the following encodings:

Operation Type CalType[2:0] TXA/TXB calibrate 000 TXA/TXB update 001TXA/TXB initialize 010 CALSET/CALTRIG calibrate 011 RXA/RXB calibrate100 RXA/RXB update 100 RXA/RXB initialize 100 Reserved 111

An additional bit (the CalType[3] bit) selects between an update to theTXA/RXA edge and the TXB/RXB edge (in other words, each calibrationoperation updates either TXA or TXB, but not both at the same time).Another bit (the CalType[4] bit) select whether PAT0 1630 or PAT1 1635pattern register is used for the calibration operation. Note that theCalType[4:3] signal would not be used during a “CALSET/CALTRIG”operation.

Once the calibration operation type has been specified, the calibrationstate machine in logic 3900 begins counting through a sequence thatproduces the appropriate pulsing of the control signals. In addition tothe control and data signals 3905 for the C3 block, the RQSel signal3960 selects the appropriate request to be placed on theCalRQ[N_(RQ)-1:0] signals 3962. The choices are based on the CALSET 1735and CALTRIG 1730 commands (which are used to update the sampling pointfor the RQ signals in each memory component), and the RDPAT0 3610,RDPAT1 2850, WRPAT0 3510, and WRPAT1 3965 commands (used to read andwrite the pattern registers in the memory components). A NOP command(not shown) is the default command selected by decode logic 1722 (FIG.17) when no other command is specified by the RQ[i,j] signal 1604.

The PatSel signal 3970 selects which of the pattern registers 1630, 1635are to be used for the operation. Pattern registers 1630 and 1635 inFIG. 37 are the pattern registers in the calibration logic 3900 thatcorrespond to the pattern registers 1630, 1635 in the memory componentwhose timing is being calibrated. Other embodiments could use adifferent number of registers in the controller and memory components.The DSel signal 3972 allows a selected pattern register to be steeredonto the D_(C)[j][3:0] signals 2060 by way of multiplexers 3976, 3978,and the RQoSel signal 3975 allows the CalRQ[N_(RQ)-1:0] signals 3962 tobe steered onto the RQ[i][N_(RQ)-1:0] signals 3920 by way of multiplexer3980. In this embodiment, all slices perform calibration operations inparallel.

During an RXA or RXB operation, the contents of a pattern register 1630or 1635 are read from a memory component and compared to the contents ofthe corresponding pattern register 1630 or 1635 in the controller.Depending upon whether the two sets of values match in the compare logic3990, the IncDecR[j][1:0] signals 3930 will be set to appropriate valuesto cause the PhaseR[j][11:0] value, such as on line 2675 of FIG. 26, tobe incremented, decremented, or left unchanged. Note that only theIncDecR[j][1:0] signals 3930, can be different from slice to slice; theother five RX[j] control signals 3925 will have the same values for allthe slices.

During a TXA or TXB calibration operation, the contents of a controllerpattern register are written to a pattern register in each memorycomponent, read back from the memory component, and compared to thecontents of the original pattern register in the controller. Dependingupon whether the two sets of values match in the compare logic 3990, theIncDecT[j][1:0] signals 3915 will be set to appropriate values to causethe PhaseT[j][11:0] value, such as on line 3375 of FIG. 33, to beincremented, decremented, or left unchanged. Note that only theIncDecT[j][1:0] signals 3915 can be different from slice to slice; theother five TX[j] control signals 3910 will have the same values for allthe slices.

There are two sets of signals that are compared by the compare logicblock 3990: Q_(C)[j][3:0] 2050 and PAT[3:0] 3995 from multiplexer 3978.The results from the compare are held in the Compare Logic 3990 when theCompareStrobe signal 3992 is pulsed. The CompareSel[3:0] signals 3998indicate which of the operation types is being executed. TheCompareSel[3] signal selects between an update to the TXA/RXA edge andthe TXB/RXB edge. The CompareSel[2:0] signals select the calibrationoperation type as follows:

Operation Type CompareSel[2:0] TXA/TXB calibrate 000 TXA/TXB update 001TXA/TXB initialize 010 reserved 011 RXA/RXB calibrate 100 RXA/RXB update100 RXA/RXB initialize 100 reserved 111

In the case of the TXA/TXB operations the IncDecT[j][1:0] signals 3915are adjusted, and in the case of the RXA/RXB operations theIncDecR[j][1:0] signals 3930 are adjusted. In the case of the updateoperations, the compare logic 3990 looks for a left or right shift ofthe read data relative to the controller pattern register, indicatingthat the memory component has changed its RQ sampling point. If there isno shift, no change is made to the phase value. In the case of thecalibrate and initialize operations, the compare logic 3990 checks ifthe read data and the controller pattern register are equal or not. Thephase value is either incremented or decremented, as a result.

The Do[j][3:0] 3901 signals are outgoing write data signals, being sentfrom elsewhere in the controller to a memory component. Similarly theRqo[i][N_(RQ)-1:0] signals are outgoing request signals, being sent fromelsewhere in the controller to a memory components. These signals arenot used by the calibration logic 3900 of the memory controller (theyare used for normal read and write operations) and thus are not furtherdiscussed here.

Alternate Embodiments

The preferred system described above with reference to FIGS. 7-37provided a complete description of a memory system topology that couldbenefit from the methods of dynamic mesochronous clocking. A number ofvariations from this baseline system will now be described individually.Any of these individual variations may, in general, be combined with anyof the others to form composite variations. Any of the alternate systemsformed from the composite variations can benefit from the methods ofdynamic mesochronous clocking.

The preferred system described routed the CLK signal on bus 1320 withthe Y bus signals (the RQ bus 1315). As a result, the CLK signal and RQbus connected to two or more memory components on a memory rank.However, two memory components in the same rank could have differenttiming parameters (t_(Bij), in particular), and as a result it may notbe possible to adjust the transmit timing of CLK and RQ signals in thecontroller so that a component of the memory system is able to receiveRQ signals 1604 with the CLKB signal 1606. This problem necessitated theuse of the RQ sampling method, in which the CLK signal 1340 has 4 timingevents (rising edges) per RQ bit time, and in which a calibrationprocess is performed to select the proper timing event for sampling. Itis possible to use more than 4 timing events and fewer than 4 timingevents to set a sampling point. For example, the lower sampling limitcan be three when calibration pulses on the RQ signals 1604 are limitedto the bit time and phase offset of normal RQ signals. In alternatepreferred embodiment, described below, the RQ calibration pulses areallowed to have phase offsets with respect to normal RQ signals.

The RQ calibration process of the preferred embodiment is very simple: ahigh pulse of duration t_(CLK1CYCLE), depicted in timing diagram 18, issampled by the CLKB signal (with a CLK4 frequency) in a component of thememory system, and looks for a string on three high values, choosing thecenter high value as the sample point. This approach can be extended tolook for a string of 3, 4, or 5 high values (the possible outcomes forall possible phase alignments of RQ and CLKB) in order to select abetter-centered sample point.

Another embodiment uses a second RQ signal for calibration. In onevariation, this second signal carries a high value during NOP commandsand carries a low pulse of duration t_(CLK1CYCLE) at the same time thatthe first RQ signal carries its high pulse. The calibration logic 1310,1355 would search the high and low sampled values to select abetter-centered sample point.

Yet another variation to the preferred embodiment is based on the factthat it is not necessary to use a CLK signal (on bus 1320) having fourrising edges per RQ bit time in order to get enough timing events toperform the sampling. It would be possible to route four separate CLKsignals on separate wires, each with one rising edge per RQ bit time,but offset in phase across the t_(CLK1CYCLE) interval. Thesephase-shifted clock signals would need to be recombined in the memorycomponent 1600 for transmitting and receiving on the DQ bus 1612. Itwould be beneficial to route multiple clock signals on a main printedcircuit board and modules so as to minimize propagation delaydifferences.

In the description of the preferred embodiment, reference was often madeto the rising edges of various clock and timing signals. But it is alsopossible to use both the rising and falling edges of a clock signal.When using both rising and falling edges, it is preferable to usedifferential signaling for the clock signal to minimize any duty cycleerror. Such differential signaling would permit the use of a clocksignal with two rising edges and two falling edges per RQ bit time,reducing the maximum frequency component of the clock signal. Thoughsuch elements are not shown, this approach would require the use ofregister elements in the memory system that use both the rising edge andfalling edge of clock.

As noted earlier, in yet another alternate embodiment it would bepossible to use a smaller number of timing events on the CLK signal perRQ bit interval. In principle, two timing events are possible (e.g., arising edge and a falling edge of a clock with cycle time oft_(CLK1CYCLE)). This alternate embodiment is performed by transmittinghigh pulses on four RQ signals, each pulse offset by ¼ of at_(CLK1CYCLE) from one another. The four pulses could be sampled by therising and falling edge of CLKB on bus 1347. The resulting pattern ofeight sample values will indicate where the bit time of the normal RQsignals lie, and the memory component can select either the rising edgeor falling edge of CLKB for sampling. This approach would requiretransmit circuitry in the controller that could adjust the relativephase of RQ signals to generate the calibration pulses. While not shown,such circuitry could be provided by one skilled in the art and based onthe requirements of a particular embodiment.

In a further variation to the preferred embodiment, it would be possibleto use more complex calibration patterns instead of, or in combinationwith, the simpler pattern consisting of a high pulse of durationt_(CLK1CYCLE). For example, a high pulse could be used to indicate thestart of the calibration sequence, with further pulses chosen to elicitthe worst case pattern sequence for the RQ receivers in the memory.These worst case patterns could be chosen at initialization time from alarger set of predefined candidate patterns and then stored for useduring calibration operations.

As noted, the preferred embodiment system routed the CLK signal with theY bus signals (the RQ bus 1315). As a result, the CLK signal 1320 and RQbus connected to two or more components on a memory rank. However, thefact that two memory components could have very different timingparameters (t_(Bij), in particular) means that it may not be possible toadjust the transmit timing of CLK and RQ in the controller so that eachmemory component is able to receive RQ with CLK. This restrictionnecessitated the use of the RQ sampling method in which the CLK signalis provided with two or more selectable timing events (edges) per RQ bittime.

If the CLK signal is routed with the X bus signals, the sampling methodis no longer necessary because each memory component in a memory rankreceives its own clock signal. The transmit timing of each CLK signalcan be adjusted in the controller so that CLKB[i,j] 1345 has the properphase to sample the RQ bus 1315. The transmit and receive timing of theDQ bus 1325 is adjusted as in the preferred embodiment.

Routing the clock signal with X bus signals on bus 1325 has the benefitthat the RQ signals can use a bit time similar to that of the DQ signalson bus 1325. However, if the RQ signals on bus 1315 connect tosignificantly more memory components of the memory system than the DQsignals, this benefit may not be fully realized since the wiringtopology may limit the maximum signaling frequency of the RQ signals.Additionally, this method may require more clock pins on the controllerif the number of slices in a rank is more than one. This increase isbecause there is one CLK signal per slice but only one CLK signal perrank with the preferred embodiment.

This method of routing the clock signal with X bus signals will alsohave a performance penalty when multiple ranks are present. This penaltyoccurs either because there are multiple ranks on one module or becausethere are multiple modules in the memory system. A read or writeoperation consists of one or more commands transferred on the RQ bus1315 followed by a data transfer on the DQ bus 1325. While commands anddata are spread out over a time interval, they are usually overlappedwith read or write operations to other independent banks of the samerank, or to banks of other ranks. But this method of routing requiresthat transfers on the RQ or DQ bus of a particular rank [j] use a CLKsignal for each slice [i] with a particular phase offset. A transfer toan RQ or DQ bus of a different rank [k] (where [k] is different than[j]) needs a CLK signal for each slice [i] that has a different phaseoffset. As a result, interleaved transfers to different ranks on the RQor DQ buses would require a settling interval between phase adjustmentsto the CLK signal for each slice impacting overall performance. Thisperformance impact could be addressed by generating a CLK signal foreach slice of each rank (a CLK signal per memory component). It couldalso be addressed by limiting the number of ranks to one, but if such alimitation on the number of ranks is not desired, a performance penaltywill result.

An alternate preferred embodiment uses the falling CLK edges to receiveand transmit data bits on the DQ bus 1325. The preferred embodiment usesthe rising edge of CLKB to receive and transmit a bit of information onthe DQ bus 1325. It is possible to use the falling edge of CLKB at 1347to also receive and transmit a DQ bit. As noted, the falling edge wouldprobably necessitate the use of differential signaling for CLK tominimize any duty cycle error. However, such an approach would equalizethe maximum frequency component of the CLK signal and the DQ signals andwould require the use of register elements in the memory components thatuse both the rising edge and falling edge of clock. It is also possibleto use the falling edge of CLKB to also receive an RQ bit from the RQbus 1315. As noted, this approach would probably necessitate the use ofdifferential signaling for CLK to minimize any duty cycle error. As withthe above case, this approach would equalize the maximum frequencycomponent of the CLK signal and the RQ signals and would require the useof register elements in the memory components that use both the risingedge and falling edge of clock.

While the description of the preferred embodiment concentrated on theoperation of the system with respect to a single rank, the timingmethods of the present invention work equally well when multiple ranksof the memory components are present. A system can have multiple rankson one module and/or multiple modules in the memory system. Thecontroller includes a storage array to store a separate set ofRXA/RXB/TXA/TXB phase adjustment values for each rank and each slice inthe system (4×12 bits of phase values for each memory component using 12bit phase resolution). The phase adjustment values for a particular rankwould be copied into the appropriate registers in the R3 (see FIG. 26)and T3 (see FIG. 33) blocks in the preferred embodiment for each slicewhen a data transfer was to be performed to that rank. It is possiblethat a settling time would be needed between data transfers to differentranks to give the phase selection and phase blending logic of the R2(see FIG. 25) and T2 (see FIG. 32) blocks for each slice time tostabilize the clocks they are generating.

The settling time needed between data transfers to different ranks willlikely impact system performance. This performance impact can beminimized by a number of techniques in the memory controller. The firstof these techniques is to perform address mapping on incoming memoryrequests. This technique involves swapping address bits of a memoryrequest so that the address bit(s) that selects an applicable rank (andmodule) come from address fields that change less frequently. Typically,these will be address bits from the upper part of the address.

The second of these techniques would be to add reordering logic to thememory controller, so that requests to a particular rank are groupedtogether and issue sequentially from the controller. In this way, thesettling time penalty for switching to a different rank can, in effect,be amortized across a larger number of requests.

If the CLK signal is routed with the X bus signals, as in a previousalternate preferred embodiment, there will need to be a separate set ofRXA/RXB/TXA/TXB register values for receiving and transmitting on the DQbus 1325 for each rank and each slice in the system. This separate setof values will require 4×12 bits of phase values with 12 bit phaseresolution. In addition, there will need to be a separate set of TXA/TXBregister values for transmitting the CLK signal for each rank and eachslice in the system (24 bits of phase values for each memory component).This embodiment requires that transfers on the RQ 1315 or DQ 1325 bus ofa particular rank [i] use a CLK signal for each slice [i] with acorresponding, separately calibrated phase offset. A transfer to an RQor DQ bus of a different rank [k] needs a CLK signal for each slice [i]that has a different phase offset. As a result, interleaved transfers todifferent ranks on the RQ or DQ buses would require a settling intervalbetween phase adjustments to the CLK signal for each slice, and asimultaneous settling interval for phase adjustments to the CLKD[0,j] orCLKQ[0,j] clock signal for each slice (see phase adjustment logic 1365and 1368 in FIG. 13). The values for a particular rank, i, would becopied into the R3 (FIG. 26) and T3 (FIG. 33) blocks for each slice, j,when a data transfer was to be performed to that rank. An additionalsettling time would likely be needed between data transfers to differentranks to give the phase selection and phase blending logic of the R2(FIG. 25) and T2 (FIG. 32) blocks for each slice time to stabilize theclocks generated.

In still a further variation, the description of the preferred systemrelated to calibration logic M4 used two registers PAT0 and PAT1(registers 1630 and 1635, respectively, FIG. 16) to hold patterns foruse in the calibration, update, and initialization sequences needed tomaintain the proper phase values in the RXA/RXB/TXA/TXB registers foreach slice. In an alternate preferred embodiment, it would be possibleto add more registers to add flexibility and robustness to thecalibration, update, and initialization sequences. Adding registerswould require at least the following: 1) enlarging the fields in thecalibration commands that select calibration registers; 2) adding moreregisters to the memory; and 3) controller hardware that performscalibration, update, and initialization sequences to ensure that theadded registers are loaded with the proper values.

In a further variation to the above, the description of the preferredembodiment used a single four bit transfer per DQ signal for performingcalibration. Since calibration operations can be pipelined (like anyread or write transfer), it would possible to generate a pattern of anylength, provided there is enough register space to hold a multicyclepattern. The use of a longer calibration pattern in an alternatepreferred embodiment ensures that the phase values used during systemoperation have more margin than in the preferred embodiment.

The preferred embodiment used the same data or test patterns forcalibrating the RXA/RXB/TXA/TXB phase values. In a further variation,one could use different patterns for each of the four phase valuesprovided there is enough register space. The use of customizedcalibration patterns for the two limits of the read and write bitwindows ensures that the phase values used during system operation areprovided with added margin.

Additionally, the preferred embodiment assumed that the calibration,update, and initialization sequences used the same patterns. However, aninitialization sequence may have access to more system resources andmore time than the other two sequences. This means that duringinitialization, many more candidate patterns can be checked, and theones that are, for example, the most conservative for eachRXA/RXB/TXA/TXB phase values for each slice can be saved in each memorycomponent for use during the calibration and update sequences.

The description of the preferred embodiment did not specify how thepatterns are placed into the pattern registers at the beginning of theinitialization sequence. Various approaches are possible here. Onepossible way would be to use sideband signals. Sideband signals aresignals that are not part of the RQ and DQ buses and which do not needthe calibration or initialization sequence to be used. Such sidebandsignals could load the pattern registers with initial pattern values.

A second possible way to accomplish placing patterns in the patternregisters is to use a static pattern that is hardwired into a read-onlypattern register. This pattern could be used to initialize the phasevalues to a usable value, and then refine the phase values withadditional patterns.

A third possible way to place patterns in the pattern registers is touse a circuit that detects when power is applied to the memorycomponent. When power is detected, the circuit could load patternregisters with initial pattern values.

A fourth possible way is to use a reset signal or command to loadpattern registers with initial pattern values.

Again returning to the preferred embodiment, the phase values in theRXA/RXB and TXA/TXB registers were averaged to give the best samplingpoint and drive point for read bits and write bits. For some systems, itmight be preferable to pick a point that is offset one way or the otherto compensate for the actual transmit and receive circuitry.

While the preferred embodiment did not explicitly show how thecalibration, update, and initialization sequences generate the controlsignals for manipulating the RXA/RXB/TXA/TXB registers, there are anumber of ways to do this. One approach is to build a state machinewhich sequences through the 60 or so cycles for the update andcalibration sequences and pulsing the control signals, as indicated inthe timing diagrams in FIGS. 29 and 34. Systems could be configured tohandle longer sequences needed for initialization. Such systems would betriggered by software, in the case of the initialization sequence, or bya timer, in the case of the update and calibration sequences. The updateand calibration sequences could arbitrate for access to the memorysystem and hold off the normal read and write requests. A second way togenerate the control signals for manipulating the RXA/RXB/TXA/TXBregisters would be to use software to schedule the sequences andgenerate the control signals. This could be an acceptable alternative insome applications.

In regards to other alternate embodiments, it was mentioned earlier thatthe dynamic mesochronous clocking techniques are suitable for systems inwhich power dissipation is important (such as portable computers). Thereare a number of methods for reducing power dissipation (at the cost ofreducing transfer bandwidth) that allow such a system to utilize anumber of different power states when power is more important thanperformance.

FIG. 38 shows an example of the logic needed in the memory controller3705 and the memory 3703 to implement a “dynamic slice width” powerreduction mechanism. In this figure, it is assumed that each memorycomponent 3703 connects to two DQ signals 3710 and 3720. This means thatthere will be two M5 blocks (3725 and 3730) and two M2 blocks (3735 and3740) inside each memory. In the figure, the two M5 blocks and two M2blocks are appended with a “−0” and “−1” designation. It is noted thatonly a read operation is discussed; a write operation would use similarblocks of logic. The internal Q_(M) signals in blocks 3725 and 3730 andexternal DQ signals 3710 and 3720 are appended with “[0]” and “[1]”.

During a normal read operation (HalfSliceWidthMode=0), the two M5 blockswill each access a four bit parallel word of read data (Q_(M)[3:0][0]and Q_(M)[3:0][1]). These data bits are converted into serial signals onDQ buses 3710 and 3720, and transmitted to the controller 3705. Theserial signals propagate to the controller where they are received(DQ[0,j][0] and DQ[0,j][1]). The R1-0 block 3745 and R1-1 block 3750 inthe controller convert the serial signals into four bit parallel words(Q_(C)[j][3:0][0] and Q_(C)[j][3:0][1]) at 3755, 3760, respectively.

During a reduced power mode (HalfSliceWidthMode=1), the clocks to theR1-1 3750 and M2-1 3740 blocks in the controller and memory aredisabled. The DQ[i,j][1] signal 3720 of each slice is not used, reducingthe available bandwidth by half. The read data from the M5-1 block 3730must be steered through the M2-0 block 3735 and onto the DQ[i,j][0]signal bus 3710. In the controller, this data must be steered from theR1-0 block 3745 onto the Q[j][3:0][1] signal bus 3760. This steering isaccomplished by a four-bit 2-to-1 multiplexer 3795 and a four bitregister 3775 in memory 3703, and by two four-bit 2-to-1 multiplexers,3770 and 3780, and a four bit register 3785 in the controller. Theselect control input of the multiplexers are driven by theHalfSliceWidthMode signal 3788. In the memory component, this signal isgated with a Load2 signal 3772 (by logic gate 3790) that alternatesbetween selecting the M5-0 read data and the M5-1 read data. The loadcontrol input of the registers are driven by the LoadR[j] signal 3795 inthe memory controller and by the Load signal 3797 in the memory 3703.

FIG. 39 shows timing signals 39(a)-(k) for a read transaction performedby the system of FIG. 38. These timing signals are similar to theprevious read transaction diagram (FIG. 14A) except where noted. TheRead command 3805 is transmitted as the RQ[i,j] signal 39(b), causing aninternal read access to be made to the two memory core blocks R5-0 andR5-1. The parallel read data Q[7:4] 3810 and Q[3:0] 3815 is available onthe two internal buses. Q_(M)[3:0][1] and Q_(M)[3:0][0] in blocks 3730and 3725, respectively, as represented in signals 39(e) and 39(f).Q[3:0] is selected first because the Load2. signal 39(d) is low and isconverted to four serial bits on the DQ[i,j][0] bus, shown as waveform39(g). Q[7:4] is selected next because the Load2 signal goes high andthis signal is also converted into four serial bits on the DQ[i,j][0]bus.

The first four bits Q[3:0] are received on the DQ[0,j][0] bus (waveform39(g)) by the R1-0 block and converted to four parallel bits, which areloaded into the register 3785. The next four bits Q[7:4] are received onthe DQ[0,j][0] bus (waveform 39(g)) by the R1-0 block and converted tofour parallel bits. These are multiplexed onto the Q_(C)[j][3:0][1]signals while the register 3785 is multiplexed onto the Q_(C)[j][3:0][0]signals by multiplexer 3770.

Note that the eight bits on the Q_(C)[j][3:0][0] and Q_(C)[j][3:0][1]signals will be valid for one cycle, and can be asserted at the maximumrate of once every two cycles. In other words, the next read transactionmust be asserted after a one cycle gap. In FIG. 39, this may be seen inthe top waveform when one READ command 3805 is asserted after CLK1 edge0 (3820), and the next READ command 3825 (with dotted outline) cannot beasserted until after CLK1 edge 2 (3830). This separation is necessarybecause each READ command transfers a total of eight bits on theDQ[i,j][0] signal, requiring two CLK1 cycles.

An alternative implementation of a reduced slice width could reduce thenumber of bits returned by each READ command, in addition to reducingthe number of DQ signals driven by each slice in block 3703. Thisapproach would have the benefit of not requiring a one CLK1 cyclebetween READ commands. Instead, this approach would require that anotheraddress bit be added to the request information on the RQ bus so thatthe one of the four bit words from the M5-0 and M5-1 blocks can bechosen. In the controller, only one of the four bit busesQ_(C)[j][3:0][0] and Q_(C)[j][3:0][1] (3775 and 3760, respectively) willcontain valid read data in each CLK1 cycle. Alternatively, if eachmemory component were connected to the controller with more than two DQsignals (four for example), then the reduced slice width modes couldinclude several slice widths.

Note that the above examples represent ways that power usage may belowered, by reducing the number of signals that each memory slicedrives. Other alternatives are also possible.

The HalfSliceWidthMode signal on the controller component and the memorycomponent would typically be driven from a storage register on thememory component, although it could also be a signal that is directlyreceived by each memory component. The HalfSliceWidthMode signal wouldbe asserted and deasserted during normal operation of the system so thatpower could be reduced. Alternatively, it could be asserted ordeasserted during initialization of the system so that power dissipationcould be set to the appropriate level. This could be important forreducing the system's temperature or for reducing the system's powerconsumption. This might be an important feature in a portable systemthat had limited cooling ability or limited battery capacity.

Note that a write transaction would use a similar set of logic in thecontroller and memory, but operating in the reverse direction. In otherwords, the multiplexer 3765 and register 3775 in the memory component ofFIG. 38 would be in the T0 cell of the controller 3705, and the twomultiplexers (3770, 3780) and register 3785 in the memory controller ofFIG. 38 would be in component 3703.

The System shown in FIG. 38 represents one way in which power might belowered by reducing the number of signals that a component in the memorysystem transmits or receives. Other methods may include reducing thenumber of signals that a rank transmits or receives, or reducing thenumber of bits transmitted or received during each read or writetransaction. These alternate methods are described below.

If individual components of memory 3703 connect to the memory controller3705 with a single DQ signal (such as 3710 and 3720 for blocks 3735 and3740), it will not be possible to offer a reduced power mode using thedynamic slice width method just described. Instead, a dynamic rank widthmethod could be used. For example, a HalfRankWidthMode signal 3788 couldbe asserted causing each read transaction or write transaction to accessonly half of the memory of the rank (e.g., with two or more memorycomponents sharing the same slice within a rank). An address bit couldbe added to the request information on the RQ bus to select between thetwo sets of memory components. Selected components would perform theaccess as in a normal read or write transaction. Memory components notselected would not perform any access and would shut off the internalclock signals as in the dynamic slice width example in FIG. 38.

Likewise, transmit and receive slices of the controller corresponding tothe selected memory components could perform the access as in a normalread or write transaction. Transmit and receive slices of the controllercorresponding to memory not selected would then not perform any accessand would shut off the internal clock signals as in the dynamic slicewidth example in FIG. 38.

In the above approaches where one is using selected and-non-selectedmemory components for power reduction, it is important to carefullychoose the address bit that selects between the two sets of memorycomponents. The address bit taken should probably come from high in thephysical address so that successive requests to the memory componentstend to access the same half-rank. The selection from high in thephysical address could be accomplished with multiplexing logic in theaddress path of the controller that selected an address bit from anumber of possible positions, possibly under the control of a value heldin a register set during system initialization.

It would also be possible to adjust the order of successive requests bypulling them out of a queue so that successive requests to the memorycomponents tend to access the same half-rank. Again, this could beaccomplished by logic in the controller. The logic would need to ensurethat out-of-order request submission produced the same results asin-order submission, permitting one of the two half-ranks to remain in alower power state for longer periods of time.

By extending the above method, it would be possible to support severalrank widths in a memory system. For example, a rank could be dividedinto quarters, requiring two address bits in the controller and eachmemory component to select the appropriate quarter-rank.

The HalfRankWidthMode signal 3788 on the controller 3705 and memorywould typically be driven from a storage register in component 3703,although it could also be a signal that is directly received bymemories. The HalfRankWidthMode signal could be asserted and deassertedduring normal operation of the memory system so that power could beeffectively reduced. Alternatively, the signal could be asserted ordeasserted during initialization of the system so that power dissipationcould be set to an optimal or appropriate level.

Reducing the number of bits that are accessed in each read or writetransaction could also reduce power. A dynamic depth mode could bedefined, in which a HalfDepthMode signal (not shown) is asserted, whichcauses each read transaction or write transaction to access only half ofthe normal number of bits for each transaction. As with a priorvariation described above, an address bit would have to be added to therequest information on the RQ bus to select between the two sets of bitsthat can be accessed. Likewise, the controller would need to use thesame address bit to decide which of the two sets of bits are beingaccessed. The transmit and receive slices of the controller and memorywould shut off the internal clock signals during the periods that nobits are being transferred. This would effectively reduce power byreducing bandwidth.

It would be possible to support several programmable depths in thesystem by extending the above HalfDepthMode method. For example, thetransfer size could be divided into quarters, requiring two address bitsin the controller and each memory component to select the appropriatequarter-transfer-block.

The HalfDepthMode signal on the controller component (such as component3705) and memory components (such as component 3703) would typically bedriven from a storage register, although it could also be a signal thatis directly received by each component. The HalfDepthMode signal wouldbe asserted and deasserted during normal operation of the system so thatpower could be reduced. Alternatively, it could be asserted ordeasserted during initialization of the system so that power dissipationcould be set to an optimal or appropriate level.

Power could also be reduced by reducing the operating frequency of thememory components. This approach is particularly appropriate for dynamicmesochronous clocking systems such as the systems described in thisdocument because there is no clock recovery circuitry in the memorycomponent. The memory component will therefore tolerate a very widerange of input clock frequency, unlike memory components that utilizesDLL or PLL circuits that typically operate in a narrow range of clockfrequencies.

A dynamic frequency mode could be defined, in which a HalfFrequencyModesignal is asserted, which caused all signals connecting the controllerand memory components to operate at half their normal signaling rate.

In reducing the operating frequency, there would be no change in amemory component such as memory component 3703, except that any timingparameter that is expressed in absolute time units (e.g., nanosecondunits, as opposed to clock cycle units) would need to be adjusted foroptimal operation. This timing parameter adjustment would typically bedone in the controller by changing the interval between commands on theRQ bus. For example, the interval between a row access and a columnaccess to that row must be greater than the t_(RCD) parameter, a corecharacteristic that is expressed in nanoseconds. The controller willtypically insert the appropriate number of clock cycles between the rowaccess command and the column access command to account for thisparameter. If the clock rate is reduced by one-half, the number ofcycles between the two commands can also be reduced by one-half. If thisreduction is not done, the memory component will still operatecorrectly, but not optimally.

The controller will also need to provide logic to manage the reductionin bandwidth of a memory system if portions of the controller are notoperated at a lower clock frequency. In other words, if the controllerruns at the normal clock rate and the memory components run at half theclock rate, then the controller will need to wait twice as long for eachmemory access. Holding registers and multiplexers can handle thisprocess using techniques similar to those for dynamic slice width inFIG. 38. It is noted that it could be possible to support severalprogrammable frequencies in the system by extending this method. Forexample, the transfer rate could be reduced to one-quarter of the normalrate.

The HalfFrequencyMode signal on the controller of a memory system wouldtypically be driven from a storage register in the controller, althoughit could also be a signal that is directly received by the controller.The HalfFrequencyMode signal would be asserted and deasserted duringnormal operation of the system so that power could be reduced.Alternatively, it could be asserted or deasserted during initializationof the system so that power dissipation could be set to an appropriatelevel.

The preferred embodiments discussed above utilize slices of memorycomponents that each had one DQ (data) signal connecting the memorycomponents in each slice to the controller. As mentioned previously withrespect to FIGS. 38 and 39, the benefits of dynamic mesochronousclocking are also realized with memory components that have widths thatare greater than one DQ signal.

For example, each memory component could have two DQ signals connectingto the memory controller. In such a system, it would be important tomaintain different sampling and driving points in the controller foreach slice of DQ signals. However, there is also some benefit tomaintaining different sampling and driving points in the controller forthe individual DQ signals within each slice.

For example, there could be some dynamic variation of the externalaccess times between the different DQ signals of one memory component.While this variation would be much smaller than the variation betweenthe DQ signals that connect to two different memory components, it ispossible that the variation would be large enough to matter. Thisvariation could be easily compensated by using an additional instance ofthe calibration circuitry described above.

Also, there is a possible static variation needed for the sampling anddriving points of the DQ signals connecting to a single memory componentbecause of differences in the length of the interconnect wires betweenthe controller and memory component. This variation could be easily“calibrated out” using an additional instance of the calibrationcircuitry described above.

Finally, it is likely that a memory controller will be designed tosupport memory components that have a variety of DQ widths, including aDQ width of one signal as well as a DQ width of two or more signals.This means that such a controller will need to be able to independentlyadjust the sample and drive points of each DQ signal. This means thatwhen memory components with a DQ width of two or more signals arepresent, the signals for each memory component can still be givendifferent sample and drive points at no extra cost.

In the preferred embodiment, within a particular rank, each slicecontains a single memory component. However, in other embodiments,within a particular rank two or more slices may be occupied by a singlememory component (i.e., where the memory component communicates with thememory controller using two or more parallel DQ signal sets). In yetother embodiments, within a particular rank a slice may contain two ormore memory components (which would therefore share a single DQ signalset and a set of calibration circuitry within the memory controller, forexample using the HalfRankWidthMode signal described above to select oneof the two memory components within each slice).

The techniques described for a memory system in accordance with thepreferred embodiments permitted phase offsets of clocked components todrift over an arbitrarily large range during system operation in orderto remove clock recovery circuits (DLL and PLL circuits) from the memorycomponents. This technique could be applied to a non-memory system justas easily, resulting in similar benefits.

For example, assume there are two logic components (integrated circuitsthat principally contain digital logic circuits, but which might alsoinclude other types of circuits including digital memory circuits andanalog circuits) that must communicate at high signaling rates. Priorart methods include placing clock recovery circuitry in both componentsto reduce timing margin lost because of timing imprecision.

Alternatively, clock recovery circuitry could be entirely removed fromone of the logic components, with all phase adjustments performed byanother component that still retains the clock recovery circuitry.Periodic calibration similar to that performed in the memory system ofthe preferred embodiment would keep the required phase offsets neartheir optimal values for communication between the two components.Keeping phase offsets near their optimal values could be important ifthere was some design or cost asymmetry between the two components. Forexample, if one component was very large, or was implemented with abetter process technology, it might make sense to place all the clockrecovery and phase adjustment circuitry in that component. Thisplacement of the circuitry in one component would allow the othercomponent to remain cheaper or to have a simpler design or use anexisting or proven design. Also, if one of the components went throughfrequent design updates, and the other component remained relativelystable, it might make sense to place all the clock recovery and phaseadjustment circuitry in the stable component.

As noted, the term “mesochronous system” refers to a set of clockedcomponents in which the clock signal for each component has the samefrequency, but can have a relative phase offset. The techniquesdescribed for a preferred system permitted the phase offsets of theclocked components to drift over an arbitrarily large range duringsystem operation in order to remove clock recovery circuits (DLL and PLLcircuits) from the memory components. If these clock recovery circuitsare left in a memory portion (i.e., not in the controller), the phaseoffsets of the memory portion will drift across a much smaller rangeduring system operation. However, such a system could still benefit fromthe techniques utilized in the preferred embodiment to maximize thesignaling rate of the data (and request) signals. In other words, insuch a system, static phase offsets for the memory components would bedetermined at system initialization. However, during system operation,these static offsets would be adjusted by small amounts to keep themcloser to their optimal points. As a result, the signaling bandwidth ofthe data (and request) signals could be higher than if these periodiccalibration operations were not carried out.

The above could be considered a pseudo-static mesochronous system sinceit will be expected that the phase offsets of the memory components willnot drift too far from the initial values. The hardware to support thiscould include all the hardware described for a system in accordance withthe preferred embodiment. However, because the dynamic phase offsetrange is expected to be smaller, it is possible that the hardwarerequired could be reduced relative to the preferred embodiment, reducingcost and design complexity.

Dynamic Mesochronous Techniques for Intra-Device Clocking andCommunication

The various techniques described permit the phase offsets of clockedcomponents to drift over an arbitrarily large range during systemoperation in order to remove clock recovery circuits (e.g., the aboveDLL and PLL circuits) from a subset of the components in the system.These techniques result in potential benefit in system cost, systempower, and system design complexity.

These techniques could also be applied to the internal blocks of asingle integrated circuit. As internal clock frequencies of integratedcircuits increase, it becomes more difficult to operate all the blocksof a device in a single synchronous clock domain. It may be advantageousto operate the blocks in a mesochronous fashion where clocks forinternal blocks are frequency-locked, but having arbitrary phases.

If the internal blocks form a static-mesochronous clocking system, thenclock-recovery circuits (such as DLL or PLL) must be present in eachblock to keep the phase locked to a static value. These clock recoverycircuits could introduce unacceptable cost in terms of area, power, ordesign complexity.

An alternative approach would be to use the dynamic mesochronoustechniques for intra-component clocking and communication (instead offor inter-component clocking and communication described in thepreferred embodiments above). When a pair of blocks communicates withone another, one block (the “master”) would send a clock signal to theother block (the “slave”). The phase difference between the clocks forthe master and slave blocks would slowly drift during the operation ofthe circuit because of temperature and supply voltage variations. Inaccordance with one alternative preferred embodiment, the master blockwould perform calibration operations to ensure that it could transmitand receive to the slave block, regardless of the current state of thephase of the slave clock. The calibration hardware and the calibrationprocess would be similar to what has been shown for the above-describedpreferred embodiments systems. Periodic calibration would keep therequired phase offsets near their optimal values for communicationbetween the two blocks.

If the clock recovery circuits are left in the slave block of theintegrated circuit, the phase offsets of the clock of the slave blockwill drift across a much smaller range during system operation. However,such a clocking arrangement could still benefit from the techniquesutilized in the preferred embodiment to maximize the signaling rate. Insuch a system, static phase offsets for the slave blocks would bedetermined at initialization. However, during operation, these staticoffsets would be adjusted by small amounts to keep them closer to theiroptimal points. As a result, signaling bandwidth could be higher thanthe case where these periodic calibration operations were not carriedout.

This intra-device clocking and communication system is similar to apseudo-static mesochronous clocking system described above, since itwill be expected that the phase offsets of the slave blocks will notdrift too far from their initial values. The hardware to support apseudo-static mesochronous device could include all the hardware for adynamic mesochronous device. However, because the dynamic phase offsetrange is expected to be smaller, it is possible that the hardwarerequired could be reduced relative to the dynamic mesochronous device,reducing cost and design complexity.

FIG. 40 shows another approach to implement the circuit for thecontroller block R2 2500. This circuit is responsible for creating theRCLK 4030 clock signal needed for receiving the read data from thememory components, and for creating the RX LD_ENA0 4032 signal forperforming serial-to-parallel conversion and the RX_LD_ENA1 4034 signalfor synchronizing receive data between the RCLK and CLK1 clock domainsin block 4100 (FIG. 41). The inputs to this circuit areCLK4BlendR[j][4:0] (line 2325), CLK4PhSelR[j][2:0] (line 2330) andCLK4CycleR[j][1:0] (line 2335) from block R3. This circuit also receivesCLK4[7:0], and CLK4Cyc[1:0] from outside of block R0 (from block C1 inFIG. 20).

The circuit for generating the RCLK 4030 signal is the same as thecircuit for generating the CLKQ[0,j] 1334 and its functionality isexplained in the description for FIG. 25.

The RX_LD_ENA0 signal indicates when the eight receive data bits havebeen serially shifted into bit registers 4110 (FIG. 41) and are ready tobe loaded onto the parallel bus 4135 (FIG. 41). The CLK4CycleR[j][1:0]signal, generated by block R3, picks one of the four possible loadpoints. When the CLK4PhSelR[j][2:1] signal equals 01 or 10, indicatingthe phase offset of the CLK4Cyc[1:0] is not within +/−90 degrees ofRCLK, then the value of the CLK4Cyc[1:0] is used directly to compute theLD_ENA_0 signal since there is sufficient setup and hold time marginsfor the sampling clock RCLK (FIG. 40). In the alternative if theCLK4PhSelR[j][2:1] signal equals 11, indicating the phase offset of theCLK4Cyc[1:0] is within −90 degrees of RCLK, then the previous value ofCLK4CYC[1:0] is sampled at the negative edge of RCLK at the latch 4014.This pre-sampled CLK4CYC[1:0] value, having sufficient setup and holdtime margins for the sampling clock RCLK, is used to compute theLD_ENA_0 signal. In the alternative if the CLK4PhSelR[j][2:1] signalequals 00, indicating the phase offset of the CLK4Cyc[1:0] is within 90degrees of RCLK, then the previous value of CLK4CYC[1:0] is sampled atthe negative edge of RCLK at the latch 4014. This pre-sampledCLK4CYC[1:0] value is then incremented by 1 by the Adder 4018. Theresultant value, having sufficient setup and hold time margins for thesampling clock RCLK, is used to compute the LD_ENA_0 signal. Finally, acomparator 4024 compares a RCLK synchronized output of the multiplexer4020, i.e., a selected CLK4Cyc[1:0] value, to the CLK4CycleR[j][1:0]value. The comparator generates a positive output, i.e. RX_LD_ENA0 isasserted, when its two inputs are equal. This signal is asserted onceevery 4 RCLK clock cycles.

The RX_LD_ENA1 signal is generated in a similar fashion as theRX_LD_ENA0 signal except the CLK4CycleR[j][1] bit is inverted before itis sent to the comparator 4026. The net effect is that the RX_LD_ENA1signal is asserted two RCLK cycles after the RX_LD_ENA0 signal isasserted.

FIG. 41 shows another approach to implement the controller block R1 2400of system 2300 for an 8-bit read data path. This circuit 4100 consistsof three stages and is responsible for receiving read data from thememory components and inserting a programmable delay. The first stage iscalled the de-serialization stage. In this stage, read data input fromthe DQ 1325 bus is converted to a parallel 8-bit bus 4135 by shiftingthe serial read data into the latches 4110 through four successive RCLKclock cycles. The latch 4108 is clocked by the negative edge of the RCLKsignal so that the even bits are latched during the negative phase ofthe RCLK signal. Meanwhile, the odd bits are latched during the positivephase of the RCLK signal.

The second stage of controller block R1 2400 is called thesynchronization stage, and is also sometimes called the skip circuit.The synchronization stage determines whether the read data is delayed byan additional two RCLK clock cycles (which is equal to a half CLK1 clockcycle), as governed by the CLK1SkipR[j] control signal. Thesynchronization stage is responsible for transferring the read data fromthe RCLK clock domain to the CLK1 clock domain, which runs at one fourthof the frequency of RCLK clock. In this stage, the latch 4140 stores theparallel read data selected by RX_LD_ENA0 signal 4032 via themultiplexer 4130. The latch 4120 stores a two-RCLK-cycles-delayed readdata selected by RX_LD_ENA1 signal 4034 via the multiplexer 4115. Theoutput of latch 4140 is coupled to an input of the multiplexer 4130 andan input of multiplexer 4115 by signal line 4112. The output of latch4120 is coupled to an input of the multiplexer 4115 by signal line 4122.

The control signal CLK1SkipR[j] selects either the output of latch 4120or latch 4140 via the multiplexer 4150 to provide the most optimal setupand hold time margins of the read data with respect to the latch 4170,which is sampled by the CLK1 signal 2015.

The final stage of controller block R1 2400 is called the levelizationstage, where a delay of zero to three CLK1 cycles is inserted into theread data path. A four-bit version of this circuit is described above indetail with respect to FIG. 24. The output of the levelization stage isthe 8-bit RDATA 4102.

FIG. 42 shows another approach to implement the controller block T23200, which is responsible for creating the TCLK clock signal 4230,TX_LD_ENA0 signal 4232 and TX_LD_EN1 signal 4234 needed for transmittingthe write data to the memory component 1310 as shown in FIG. 30. Thiscircuit is exactly the same as the one described in FIG. 40 except theinputs to this circuit are CLK4BlendT[j][4:0], CLK4PhSelT[j][2:0] andCLK4CycleT[j][1:0] from block T3. This circuit also receives CLK4[7:0]and CLK4Cyc[1:0] from outside of block T0 (from block C1 in FIG. 21).

FIG. 43 shows another approach for implementing the controller block T13100, which is responsible for transmitting write data on an 8-bitparallel bus 4302 to memory and inserting a programmable delay. Similarto the receive read data path, this circuit also consists of threestages, namely levelization, synchronization and serialization.

The first stage of levelization is the same as the embodiment shown inFIG. 31, except that in this embodiment the data path is eight bits wideinstead of four bits wide.

In the synchronization stage, the write data is written into and thensent from latch 4355 in the CLK1 domain. This write data is selected viathe multiplexer 4350 by the TX_LD_ENA1 signal 4234 prior to beingsampled by the TCLK signal 4230 and stored in latch 4320. TheCLK1SkipT[j] selects either the output of latch 4320 (TX_LD_ENA1-delayedwrite data) or the output of latch 4355 via multiplexer 4340 to providethe most optimal setup and hold time margins of the write data withrespect to the latches 4304, which are sampled by the TCLK signal 4230.

The final serialization stage is similar to the embodiment described inFIG. 31 except that two parallel sets of 4-bit shift registers 4304 areused to store the 8 bit parallel write data. Six of the write data bits,0 through 5, are independently loaded via a set of 2-to-1 multiplexers4306 controlled by the TX_LD_ENA0 signal 4232. Another multiplexer 4308is controlled by the TCLK, which alternatively selects an even bitduring the positive phase of the TCLK and an odd bit during the negativephase of the TCLK. The selected transmit write data bit is sent to thememory via DQ bus 1325.

The data signals on the DQ bus 1325 may be transmitted and received aseither single ended or differential data signals. In other embodiments,the number of bits transmitted through the T1 and R1 circuits duringeach CLK1 clock cycle may be fewer or greater than in the embodimentsdescribed above. Further, in other embodiments the ratio of the RCLK andTCLK clock rates to the CLK1 clock rate could be greater than or lessthan the four-to-one clock rate ratio used in the preferred embodiments.For instance, clock rate ratios of two or eight might be used in otherembodiments.

While the present invention has been described with reference to a fewspecific embodiments, the description is illustrative of the inventionand is not to be construed as limiting the invention. Variousmodifications may occur to those skilled in the art without departingfrom the true spirit and scope of the invention as defined by theappended claims.

1. A memory system, comprising: a memory controller; a memory componenthaving a memory core for holding read data information and an interfacethat is coupled to at least one bus that is also coupled to the memorycontroller for conveying signals between the memory controller and thememory component; the interface of the memory component configured to:receive a first signal from the memory controller with read requestinformation; retrieve the read data information from the memory core inresponse to the request information; and transmit to the memorycontroller a second signal containing the read data information, whereinthe read data information is comprised of read data symbols and wherethe average duration of the read data symbols, measured at theinterface, defines a symbol time interval, wherein the read requestinformation includes a first read request and a second read requesthaving an internal access time similar to an internal read access timeof the first read request; wherein operation of the memory component ischaracterized by: a first external access time interval, measured at theinterface, between the first read request and respective first read datatransmitted by the interface in response to the first read request; asecond external access time interval, measured at the interface, betweenthe second read request and respective second read data transmitted bythe interface in response to the second read request; and the differencebetween the first external access time and the second external accesstime is greater than one-half of the symbol time interval.
 2. The systemof claim 1, wherein the difference between the first access time and thesecond access time is greater than one symbol time interval.
 3. Thesystem of claim 1, wherein the operation of the memory component isfurther characterized by: a third external access time interval,measured at the interface, between a third read request and respectivethird read data transmitted by the interface in response to the thirdread request; the difference between the first external access time andthe third external access time is less than one-half of the symbol timeinterval.
 4. The system of claim 1, wherein the interface of the memorycomponent is further configured to receive timing signals from thememory controller, the timing signals carrying timing informationindicating when the read request information is to be sampled and whenthe read data symbols are to be driven.
 5. A memory system, comprising:a memory controller; a memory component having a memory core for holdingread data information and an interface that is coupled to at least onebus that is also coupled to the memory controller for conveying signalsbetween the memory controller and the memory component; the interface ofthe memory component configured to: receive a timing signal; receive afirst signal with read request information; retrieve the read datainformation from the memory core in response to the request information;and transmit a second signal containing the read data information,wherein the read data information is comprised of read data symbols andwhere the average duration of the read data symbols, measured at theinterface, defines a symbol time interval, wherein the read requestinformation includes a first read request and a second read requesthaving an internal access time similar to an internal read access timeof the first read request; wherein operation of the memory component ischaracterized by: a first external access time interval, measured at theinterface, between a timing event on the timing signal that isassociated with the first read request and respective first read datatransmitted by the interface in response to the first read request; asecond external access time interval, measured at the interface, betweena timing event on the timing signal that is associated with the secondread request and respective second read data transmitted by theinterface in response to the second read request; and the differencebetween the first external access time and the second external accesstime is greater than one-half of the symbol time interval.
 6. The systemof claim 5, wherein the difference between the first access time and thesecond access time is greater than one symbol time interval.
 7. Thesystem of claim 5, wherein the operation of the memory component isfurther characterized by: a third external access time interval,measured at the interface, between a third read request and respectivethird read data transmitted by the interface in response to the thirdread request; the difference between the first external access time andthe third external access time is less than one-half of the symbol timeinterval.
 8. A memory system, comprising: a memory controller; a memorycomponent having a memory core for holding read data information and aninterface that is coupled to at least one bus that is also coupled tothe memory controller for conveying signals between the memorycontroller and the memory component; the interface of the memorycomponent configured to: receive a first signal with timing information;retrieve the read data information from the memory core; and transmit asecond signal containing the read data information, wherein the readdata information is comprised of read data symbols and where the averageduration of the read data symbols, measured at the interface, defines asymbol time interval; wherein operation of the memory component ischaracterized by: a first drive offset time interval, measured at theinterface, between a first timing event on the timing signal and arespective first read data symbol transmitted by the interface, thefirst timing event associated with the first read data symbol; a seconddrive offset time interval, measured at the interface, between a secondtiming event on the timing signal and a respective second read datasymbol transmitted by the interface, the second timing event associatedwith the second read data symbol; and the difference between the firstdrive offset time and the second drive offset time is greater thanone-half of the symbol time interval.
 9. A system, comprising: a firstcomponent; a second component, the second component including storagefor holding data information and an interface that is coupled to atleast one bus that is also coupled to the first component for conveyingsignals between the first component and the second component; theinterface of the second component configured to: receive a timingsignal; retrieve the data information from the storage; and transmit asecond signal containing the data information, wherein the datainformation is comprised of data symbols and where the average durationof the data symbols, measured at the interface, defines a symbol timeinterval; wherein operation of the second component is characterized by:a first drive offset time interval, measured at the interface, between afirst timing event on the timing signal and a respective first datasymbol transmitted by the interface, the first timing event associatedwith the first data symbol; a second drive offset time interval,measured at the interface, between a second timing event on the timingsignal and a respective second read data symbol transmitted by theinterface, the second timing event associated with the second datasymbol; and the difference between the first drive offset time and thesecond drive offset time is greater than one-half of the symbol timeinterval.
 10. The system of claim 9 where the second component includesan internal module configured to perform functions other than memoryfunctions.
 11. A device comprising a single integrated circuit having aplurality of internal modules, the device comprising: a first module;and a second module having storage for holding data information and aninterface that is coupled to at least one bus that is also coupled tothe first module for conveying signals between the first module and thesecond module; the interface of the second module configured to: receivea first signal with timing information; retrieve the data informationfrom the storage; and transmit a second signal containing the datainformation, wherein the data information is comprised of data symbolsand where the average duration of the data symbols, measured at theinterface, defines a symbol time interval; wherein operation of thecomponent is characterized by: a first drive offset time interval,measured at the interface, between a first timing event on the timingsignal and a respective first data symbol transmitted by the interface,the first timing event associated with the first data symbol; a seconddrive offset time interval, measured at the interface, between a secondtiming event on the timing signal and a respective second data symboltransmitted by the interface, the second timing event associated withthe second data symbol; and the difference between the first driveoffset time and the second drive offset time is greater than one-half ofthe symbol time interval.
 12. A memory component comprising: a memorycore for holding read data information; and an interface configured to:receive a first signal with read request information; retrieve the readdata information from the memory core in response to the requestinformation; and transmit a second signal containing the read datainformation, wherein the read data information is comprised of read datasymbols and where the average duration of the read data symbols,measured at the interface, defines a symbol time interval, wherein theread request information includes a first read request and a second readrequest having an internal access time similar to an internal readaccess time of the first read request; wherein operation of the memorycomponent is characterized by: a first external access time interval,measured at the interface, between the first read request and respectivefirst read data transmitted by the interface in response to the firstread request; a second external access time interval, measured at theinterface, between the second read request and respective second readdata transmitted by the interface in response to the second readrequest; and the difference between the first external access time andthe second external access time is greater than one-half of the symboltime interval.
 13. The memory component of claim 12, wherein thedifference between the first external access time and the secondexternal access time is greater than one symbol time interval.
 14. Thememory component of claim 12, wherein the interface of the memorycomponent is further configured to receive timing signals, the timingsignals carrying timing information indicating when the read requestinformation is to be sampled and when the read data symbols are to bedriven.
 15. A memory component comprising: a memory core for holdingread data information; and an interface configured to: receive a timingsignal; receive a first signal with read request information; retrievethe read data information from the memory core in response to therequest information; and transmit a second signal containing the readdata information, wherein the read data information is comprised of readdata symbols and where the average duration of the read data symbols,measured at the interface, defines a symbol time interval, wherein theread request information includes a first read request and a second readrequest having an internal access time similar to an internal readaccess time of the first read request; wherein operation of the memorycomponent is characterized by: a first external access time interval,measured at the interface, between a timing event on the timing signalthat is associated with the first read request and respective first readdata transmitted by the interface in response to the first read request;a second external access time interval, measured at the interface,between a timing event on the timing signal that is associated with thesecond read request and respective second read data transmitted by theinterface in response to the second read request; and the differencebetween the first external access time and the second external accesstime is greater than one-half of the symbol time interval.
 16. Thememory component of claim 15, wherein the difference between the firstexternal access time and the second external access time is greater thanone symbol time interval.
 17. A memory component comprising: a memorycore for holding read data information; and an interface configured to:receive a timing signal; retrieve the read data information from thememory core; and transmit a second signal containing the read datainformation, wherein the read data information is comprised of read datasymbols and where the average duration of the read data symbols,measured at the interface, defines a symbol time interval; whereinoperation of the memory component is characterized by: a first driveoffset time interval, measured at the interface, between a first timingevent on the timing signal and a respective first read data symboltransmitted by the interface, the first timing event associated with thefirst read data symbol; a second drive offset time interval, measured atthe interface, between a second timing event on the timing signal andrespective second read data symbol transmitted by the interface, thesecond timing event associated with the second read data symbol; and thedifference between the first drive offset time and the second driveoffset time is greater than one-half of the symbol time interval.
 18. Adevice comprising a single integrated circuit having a plurality ofmodules, the device comprising: a non-memory module; storage for holdingdata information; and an interface configured to: receive a timingsignal; retrieve the data information from the storage; and transmit asecond signal containing the data information, wherein the datainformation is comprised of data symbols and where the average durationof the data symbols, measured at the interface, defines a symbol timeinterval; wherein operation of the electronic device is characterizedby: a first drive offset time interval, measured at the interface,between a first timing event on the timing signal and respective firstdata symbol transmitted by the interface, the first timing eventassociated with the first data symbol; a second drive offset timeinterval, measured at the interface, between a second timing event onthe timing signal and respective second read data symbol transmitted bythe interface, the second timing event associated with the second datasymbol; and the difference between the first drive offset time and thesecond drive offset time is greater than one-quarter of the symbol timeinterval.
 19. A memory system, comprising: a memory controller; a memorycomponent having a memory core for holding data and an interface that iscoupled to at least one bus that is also coupled to the memorycontroller for conveying signals between the memory controller and thememory component; the interface of the memory component configured to:receive a timing signal; receive a first signal from the memorycontroller with write request information; receive a second signal fromthe memory controller with write data information; store the write datainformation in the memory core in response to the write requestinformation; and wherein the write data information is comprised of datasymbols and where the average duration of the data symbols, measured atthe interface, defines a symbol time interval, wherein the write requestinformation includes a first write request and a second write requesthaving an internal access time similar to an internal write access timeof the first write request; wherein operation of the memory component ischaracterized by: a first external access time interval, measured at theinterface of the memory component, between the first write request and arespective first write data symbol received by the memory component inconjunction with the first write request; a second external access timeinterval, measured at the interface of the memory component, between thesecond write request and a respective second write data symbol receivedby the memory component in conjunction with the second write request;and the difference between the first external access time interval andthe second external access time interval is greater than one-half of thesymbol time interval.
 20. The system of claim 19, wherein the differencebetween the first external access time and the second external accesstime is greater than one symbol time interval.
 21. The system of claim19, wherein the interface of the memory component is further configuredto receive timing signals from the memory controller, the timing signalscarrying timing information indicating when the write requestinformation is to be sampled and when the write data symbols are to besampled.
 22. A memory system, comprising: a memory controller; a memorycomponent having a memory core for holding data and an interface that iscoupled to at least one bus that is also coupled to the memorycontroller for conveying signals between the memory controller and thememory component; the interface of the memory component configured to:receive a first signal from the memory controller with write requestinformation; receive a second signal from the memory controller withwrite data information; store the write data information in the memorycore in response to the write request information; and wherein the writedata information is comprised of data symbols and where the averageduration of the data symbols, measured at the interface, defines asymbol time interval, wherein the write request information includes afirst write request and a second write request having an internal accesstime similar to an internal write access time of the first writerequest; wherein operation of the memory component is characterized by:a first external access time interval, measured at the interface of thememory component, between a first timing event on the timing signal anda respective first write data symbol received by the memory component inconjunction with the first write request, the first timing eventassociated with the first write request; a second external access timeinterval, measured at the interface of the memory component, between asecond timing event on the timing signal and a respective second writedata symbol received by the memory component in conjunction with thesecond write request, the second timing event associated with the secondwrite request; and the difference between the first external access timeinterval and the second external access time interval is greater thanone-half of the symbol time interval.
 23. The system of claim 22,wherein the difference between the first external access time intervaland the second external access time interval is greater than one symboltime interval.
 24. A memory system, comprising: a memory controller; amemory component having a memory core for holding data and an interfacethat is coupled to at least one bus that is also coupled to the memorycontroller for conveying signals between the memory controller and thememory component; the interface of the memory component configured to:receive a timing signal; receive a first signal from the memorycontroller with write request information; receive a second signal fromthe memory controller with write data information; store the write datainformation in the memory core in response to the write requestinformation; and wherein the write data information is comprised of datasymbols and where the average duration of the data symbols, measured atthe interface, defines a symbol time interval, wherein the write requestinformation includes a first write request and a second write requesthaving an internal access time similar to an internal write access timeof the first write request; wherein operation of the memory component ischaracterized by: a first offset time interval, measured at theinterface, between a first timing event on the timing signal and arespective first write data symbol received by the interface, the firsttiming event associated with the first write data symbol; a secondoffset time interval, measured at the interface, between a second timingevent on the timing signal and a respective second write data symbolreceived by the interface, the second timing event associated with thesecond write data symbol; and the difference between the first offsettime and the second offset time is greater than one-half of the symboltime interval.
 25. A system, comprising: a first component; a secondcomponent, the second component including storage for holding data andan interface that is coupled to at least one bus that is also coupled tothe first component for conveying signals between the first componentand the second component; the interface of the second componentconfigured to: receive a timing signal; receive a second signal from thefirst component with write data information; store the write datainformation in the storage; and wherein the write data information iscomprised of data symbols and where the average duration of the datasymbols, measured at the interface, defines a symbol time interval;wherein operation of the second component is characterized by: a firstoffset time interval, measured at the interface, between a first timingevent on the timing signal and a respective first write data symbolreceived by the interface, the first timing event associated with thefirst write data symbol; a second offset time interval, measured at theinterface, between a second timing event on the timing signal and arespective second write data symbol received by the interface, thesecond timing event associated with the second write data symbol; andthe difference between the first offset time and the second offset timeis greater than one-half of the symbol time interval.
 26. A devicecomprising a single integrated circuit having a plurality of modules,the device comprising: a first module comprising a controller; and asecond module having a memory core for holding data and an interfacethat is coupled to at least one bus that is also coupled to the memorycontroller for conveying signals between the memory controller and thememory component; the interface of the second module configured to:receive a timing signal; receive a first signal from the controller withwrite request information; receive a second signal from the controllerwith write data information; store the write data information in thememory core in response to the write request information; and whereinthe write data information is comprised of data symbols and where theaverage duration of the data symbols, measured at the interface, definesa symbol time interval, wherein the write request information includes afirst write request and a second write request having an internal accesstime similar to an internal write access time of the first writerequest; wherein operation of the memory component is characterized by:a first offset time interval, measured at the interface, between a firsttiming event on the timing signal and a respective first write datasymbol received by the interface, the first timing event associated withthe first write data symbol; a second offset time interval, measured atthe interface, between a second timing event on the timing signal and arespective second write data symbol received by the interface, thesecond timing event associated with the second write data symbol; andthe difference between the first offset time and the second offset timeis greater than one-half of the symbol time interval.
 27. A memorycomponent comprising: a memory core for holding data; an interfaceconfigured to: receive a first signal with write request information;receive a second signal with write data information; store the writedata information in the memory core in response to the write requestinformation; and wherein the write data information is comprised of datasymbols and where the average duration of the data symbols, measured atthe interface, defines a symbol time interval, wherein the write requestinformation includes a first write request and a second write requesthaving an internal access time similar to an internal write access timeof the first write request; wherein operation of the memory component ischaracterized by: a first external access time interval, measured at theinterface of the memory component, between the first write request and arespective first write data symbol received by the memory component inconjunction with the first write request; a second external access timeinterval, measured at the interface of the memory component, between thesecond write request and a respective first write data symbol receivedby the memory component in conjunction with the second write request;and the difference between the first external access time and the secondexternal access time is greater than one-half of the symbol timeinterval.
 28. The memory component of claim 27, wherein the differencebetween the first access time and the second access time is greater thanone symbol time interval.
 29. The memory component of claim 27, whereinthe interface of the memory component is further configured to receivetiming signals, the timing signals carrying timing informationindicating when the write request information is to be sampled and whenthe write data symbols are to be sampled.
 30. A memory componentcomprising: a memory core for holding data; an interface configured to:receive a timing signal; receive a first signal with write requestinformation; receive a second signal with write data information; storethe write data information in the memory core in response to the writerequest information; and wherein the write data information is comprisedof data symbols and where the average duration of the data symbols,measured at the interface, defines a symbol time interval, wherein thewrite request information includes a first write request and a secondwrite request having an internal access time similar to an internalwrite access time of the first write request; wherein operation of thememory component is characterized by: a first external access timeinterval, measured at the interface of the memory component, between afirst timing event on the timing signal and a respective first writedata symbol received by the memory component in conjunction with thefirst write request, the first timing event associated with the firstwrite request; a second external access time interval, measured at theinterface of the memory component, between a second timing event on thetiming signal and a respective second write data symbol received by thememory component in conjunction with the second write request, thesecond timing event associated with the second write request; and thedifference between the first external access time and the secondexternal access time is greater than one-half of the symbol timeinterval.
 31. The memory component of claim 30, wherein the differencebetween the first external access time and the second external accesstime is greater than one symbol time interval.
 32. A memory component,comprising: a memory core for holding data and an interface that iscoupled to at least one bus that is also coupled to the memorycontroller for conveying signals between the memory controller and thememory component; the interface of the memory component configured to:receive a timing signal; receive a second signal with write datainformation to be stored in the memory core; and store the write datainformation in the memory core; wherein the write data information iscomprised of write data symbols and where the average duration of thewrite data symbols, measured at the interface, defines a symbol timeinterval; wherein operation of the memory component is characterized by:a first offset time interval, measured at the interface, between a firsttiming event on the timing signal and a respective first write datasymbol received by the interface, the first timing event associated withthe first write data symbol; a second offset time interval, measured atthe interface, between a second timing event on the timing signal and arespective second write data symbol received by the interface, thesecond timing event associated with the second write data symbol; andthe difference between the first offset time and the second offset timeis greater than one-half of the symbol time interval.
 33. A devicecomprising a single integrated circuit having a plurality of modules,the device comprising: storage for holding data; an interface coupled tothe storage, the interface configured to: receive a timing signal;receive a second signal with write data information; and store the writedata information in the storage; wherein the write data information iscomprised of write data symbols and where the average duration of thewrite data symbols, measured at the interface, defines a symbol timeinterval; wherein operation of the second component is characterized by:a first offset time interval, measured at the interface, between a firsttiming event on the timing signal and a respective first write datasymbol received by the interface, the first timing event associated withthe first write data symbol; a second offset time interval, measured atthe interface, between a second timing event on the timing signal and arespective second write data symbol received by the interface, thesecond timing event associated with the second write data symbol; andthe difference between the first offset time and the second offset timeis greater than one-half of the symbol time interval.
 34. A memorysystem, comprising: a memory controller; a first memory component havingan interface coupled to the memory controller, the interface configuredto receive respective read data symbols from the first memory component,where average duration of the respective read data symbols, measured atthe interface of the first memory component, defines a symbol timeinterval; the interface of the first memory component also configured toreceive from the memory controller respective request signals, includinga first read request; and a second memory component having an interfacecoupled to the memory controller, the interface configured to receivesecond read data symbols from the second memory component; the interfaceof the second memory component also configured to receive from thememory controller respective request signals, including a second readrequest; a first external access time interval, measured at theinterface of the first memory component, between the first read requestand a respective first read data symbol retrieved by the first memorycomponent in response to the first read request; a second externalaccess time interval, measured at the interface of the second memorycomponent, between the second read request and a respective second readdata symbol retrieved by the second memory component in response to thesecond read request; and wherein the first and second memory componentsare coupled to the memory controller by different data buses and by acommon request bus that conveys the respective request signals, and thedifference between the first external access time and the secondexternal access time is greater than one-half of the symbol timeinterval.
 35. The system of claim 34, wherein the difference between thefirst external access time and the second external access time isgreater than one symbol time interval.
 36. The system of claim 34,wherein the interface of the first memory component and the interface ofthe second memory component are each further configured to receivetiming signals from the memory controller, the timing signals carryingtiming information indicating when the read request information is to besampled and when the read data symbols are to be driven.
 37. A memorysystem, comprising: a memory controller; a first memory component havingan interface coupled to the memory controller, the interface configuredto receive respective read data symbols from the first memory component,where average duration of the respective read data symbols, measured atthe interface of the first memory component, defines a symbol timeinterval; the interface of the first memory component also configured toreceive from the memory controller a timing signal and respectiverequest signals, the request signals including a first read request; anda second memory component having an interface coupled to the memorycontroller, the interface configured to receive second read data symbolsfrom the second memory component; the interface of the second memorycomponent also configured to receive from the memory controller thetiming signal and respective request signals, including a second readrequest; a first external access time interval, measured at theinterface of the first memory component, between a first timing event onthe timing signal and a respective first read data symbol retrieved bythe first memory component in response to the first read request, thefirst timing event associated with the first read request; a secondexternal access time interval, measured at the interface of the secondmemory component, between the first timing event on the timing signaland a respective second read data symbol retrieved by the second memorycomponent in response to the first read request, the second timing eventassociated with the first read request; and wherein the first and secondmemory components are coupled to the memory controller by different databuses and by a common request bus that conveys the respective requestsignals, and the difference between the first external access timeinterval and the second external access time interval is greater thanone-half of the symbol time interval.
 38. The memory system of claim 37,wherein the difference between the first external access time intervaland the second external access time interval is greater than one symboltime interval.
 39. A memory system, comprising: a memory controller; afirst memory component having an interface coupled to the memorycontroller, the interface configured to receive respective write datasymbols from the memory controller, where average duration of therespective write data symbols, measured at the interface of the firstmemory component, defines a symbol time interval; the interface of thefirst memory component also configured to receive from the memorycontroller respective write request signals, including a first writerequest; and a second memory component having an interface that iscoupled to the memory controller, the interface of the second memorycomponent configured to receive respective write data symbols from thememory controller; the interface of the second memory component alsoconfigured to receive from the memory controller respective writerequest signals, including a second write request; and a first externalaccess time, measured at the interface of the first memory component,between the first write request and a respective first write data symbolreceived by the first memory component in conjunction with the firstwrite request; a second external access time, measured at the interfaceof the second memory component, between the first write request and arespective second write data symbol retrieved by the second memorycomponent in response to the first write request; and the differencebetween the first external access time and the second external accesstime is greater than one-half of the symbol time interval.
 40. Thememory system of claim 39, wherein the difference between the firstexternal access time and the second external access time is greater thanone symbol time interval.
 41. The memory system of claim 39, wherein theinterface of the first memory component and the interface of the secondmemory component are each further configured to receive timing signalsfrom the memory controller, the timing signals carrying timinginformation indicating when the write request information is to besampled and when the write data symbols are to be sampled.
 42. A memorysystem, comprising: a memory controller; a first memory component havingan interface coupled to the memory controller, the interface configuredto receive respective write data symbols from the memory controller,where average duration of the respective write data symbols, measured atthe interface of the first memory component, defines a symbol timeinterval; the interface of the first memory component also configured toreceive from the memory controller respective timing signal and writerequest signals, including a first write request; and a second memorycomponent having an interface that is coupled to the memory controller,the interface of the second memory component configured to receiverespective write data symbols from the memory controller; the interfaceof the second memory component also configured to receive from thememory controller the timing signals and respective write requestsignals, including a second write request; and a first external accesstime, measured at the interface of the first memory component, between afirst timing event on the timing signal and a respective first writedata symbol received by the first memory component in conjunction withthe first write request, the first timing event associated with thefirst write request; a second external access time, measured at theinterface of the second memory component, between the first timing eventon the timing signal and a respective second write data symbol retrievedby the second memory component in response to the second write request;and the difference between the first external access time and the secondexternal access time is greater than one-half of the symbol timeinterval.
 43. The memory system of claim 42, wherein the differencebetween the first external access time and the second external accesstime is greater than one of the symbol time interval.
 44. A memorysystem, comprising: a memory controller having an interface that iscoupled to a first data bus and a second data bus; and a memorycomponent having an interface that is coupled to the first data bus andthe second data bus; the system having a first mode of operation inwhich a first memory access causes a first data symbol associated with afirst memory location to be transferred on the first data bus and causesa second data symbol associated with a second memory location to betransferred on the second data bus; and the system having a second modeof operation in which a second memory access causes the first datasymbol associated with the first memory location to be transferred onthe first data bus and at a later time a third memory access causes thesecond data symbol associated with the second memory location to betransferred on the first data bus.
 45. The memory system of claim 44,wherein the system is configured to change between the first mode ofoperation and the second mode of operation between memory accesses. 46.A memory controller comprising: an interface that is coupled to a firstdata bus and a second data bus; and logic having a first mode ofoperation and a second mode of operation, the first mode of operationcomprising a mode of operation in which a first memory access causes afirst data symbol associated with a first memory location to betransferred on the first data bus and causes a second data symbolassociated with a second memory location to be transferred on the seconddata bus; and the second mode of operation comprising a mode ofoperation in which a second memory access causes the first data symbolassociated with the first memory location to be transferred on the firstdata bus and at a later time a third memory access causes the seconddata symbol associated with the second memory location to be transferredon the first data bus.
 47. The memory controller of claim 46, whereinthe logic is configured to change between the first mode of operationand the second mode of operation between memory accesses.
 48. A memorycomponent comprising, a memory core for holding read data information;an interface that is coupled to a first data bus and a second data bus;and logic having a first mode of operation and a second mode ofoperation, the first mode of operation comprising a mode of operation inwhich a first memory access causes a first data symbol associated with afirst memory location in the memory core to be transferred on the firstdata bus and causes a second data symbol associated with a second memorylocation in the memory core to be transferred on the second data bus;and the second mode of operation comprising a mode of operation in whicha second memory access causes the first data symbol associated with thefirst memory location to be transferred on the first data bus and at alater time a third memory access causes the second data symbolassociated with the second memory location to be transferred on thefirst data bus.
 49. The memory component of claim 48, wherein the logicis configured to change between the first mode of operation and thesecond mode of operation between memory accesses.
 50. A memory system,comprising: a memory controller having an interface that is coupled to afirst data bus and a second data bus; and a memory component having aninterface that is coupled to the first data bus and the second data bus;the system having a first mode of operation in which a first memoryaccess causes a first data symbol associated with a first memorylocation to be transferred on the first data bus and causes a seconddata symbol associated with a second memory location to be transferredon the second data bus; and the system having a second mode of operationin which a second memory access causes the first data symbol associatedwith the first memory location to be transferred on the first data busand causes the second data symbol associated with the second memorylocation to be transferred on the first data bus after the first datasymbol has been transferred on the first data bus.
 51. The memory systemof claim 50, wherein the system is configured to change between thefirst mode of operation and the second mode of operation between memoryaccesses.
 52. A memory controller comprising: an interface that iscoupled to a first data bus and a second data bus; and logic having afirst mode of operation and a second mode of operation, the first modeof operation comprising a mode of operation in which a first memoryaccess causes a first data symbol associated with a first memorylocation to be transferred on the first data bus and causes a seconddata symbol associated with a second memory location to be transferredon the second data bus; and the second mode of operation comprising amode of operation in which a second memory access causes the first datasymbol associated with the first memory location to be transferred onthe first data bus and causes the second data symbol associated with thesecond memory location to be transferred on the first data bus after thefirst data symbol has been transferred on the first data bus.
 53. Thememory controller of claim 52, wherein the logic is configured to changebetween the first mode of operation and the second mode of operationbetween memory accesses.
 54. A memory component comprising, a memorycore for holding read data information; an interface that is coupled toa first data bus and a second data bus; and logic having a first mode ofoperation and a second mode of operation, the first mode of operationcomprising a mode of operation in which a first memory access causes afirst data symbol associated with a first memory location in the memorycore to be transferred on the first data bus and causes a second datasymbol associated with a second memory location in the memory core to betransferred on the second data bus; and the second mode of operationcomprising a mode of operation in which a second memory access causesthe first data symbol associated with the first memory location to betransferred on the first data bus and causes the second data symbolassociated with the second memory location to be transferred on thefirst data bus after the first data symbol has been transferred on thefirst data bus.
 55. The memory component of claim 54, wherein the logicis configured to change between the first mode of operation and thesecond mode of operation between memory accesses.
 56. A memory system,comprising: a memory controller having an interface that is coupled to afirst data bus; and a memory component having an interface that iscoupled to the first data bus; the system having a first mode ofoperation in which a first memory access causes a first data symbolassociated with a first memory location to be transferred on the firstdata bus and causes a second data symbol associated with a second memorylocation to be transferred on the first data bus after the first datasymbol has been transferred; and the system having a second mode ofoperation in which a second memory access causes the first data symbolassociated with the first memory location to be transferred on the firstdata bus and at a later time a third memory access causes the seconddata symbol associated with the second memory location to be transferredon the first data bus.
 57. The memory system of claim 56, wherein thesystem is configured to change between the first mode of operation andthe second mode of operation between memory accesses.
 58. A memorycontroller comprising: an interface that is coupled to a first data bus;and logic having a first mode of operation and a second mode ofoperation, the first mode of operation comprising a mode of operation inwhich a first memory access causes a first data symbol associated with afirst memory location to be transferred on the first data bus and causesa second data symbol associated with a second memory location to betransferred on the first data bus after the first data symbol has beentransferred; and the second mode of operation comprising a mode ofoperation in which a second memory access causes the first data symbolassociated with the first memory location to be transferred on the firstdata bus and at a later time a third memory access causes the seconddata symbol associated with the second memory location to be transferredon the first data bus.
 59. The memory controller of claim 58, whereinthe logic is configured to change between the first mode of operationand the second mode of operation between memory accesses.
 60. A memorycomponent comprising, a memory core for holding read data information;an interface that is coupled to a first data bus; and logic having afirst mode of operation and a second mode of operation, the first modeof operation comprising a mode of operation in which a first memoryaccess causes a first data symbol associated with a first memorylocation to be transferred on the first data bus and causes a seconddata symbol associated with a second memory location to be transferredon the first data bus after the first data symbol has been transferred;and the second mode of operation comprising a mode of operation in whicha second memory access causes the first data symbol associated with thefirst memory location to be transferred on the first data bus and at alater time a third memory access causes the second data symbolassociated with the second memory location to be transferred on thefirst data bus.
 61. The memory component of claim 60, wherein the logicis configured to change between the first mode of operation and thesecond mode of operation between memory accesses.
 62. A memory system,comprising: a memory controller having an interface that is coupled to afirst data bus and a second data bus; and a memory module having aninterface that is coupled to the first data bus and the second data bus;the system having a first mode of operation in which a first memoryaccess causes a first data symbol associated with a first memorylocation to be transferred on the first data bus and causes a seconddata symbol associated with a second memory location to be transferredon the second data bus; and the system having a second mode of operationin which a second memory access causes the first data symbol associatedwith the first memory location to be transferred on the first data busand at a later time a third memory access causes the second data symbolassociated with the second memory location to be transferred on thesecond data bus.
 63. The memory system of claim 62, wherein the systemis configured to change between the first mode of operation and thesecond mode of operation between memory accesses.
 64. A memorycontroller comprising: an interface that is coupled to a first data busand a second data bus; and logic having a first mode of operation and asecond mode of operation, the first mode of operation comprising a modeof operation in which a first memory access causes a first data symbolassociated with a first memory location to be transferred on the firstdata bus and causes a second data symbol associated with a second memorylocation to be transferred on the second data bus; and the second modeof operation comprising a mode of operation in which a second memoryaccess causes the first data symbol associated with the first memorylocation to be transferred on the first data bus and at a later time athird memory access causes the second data symbol associated with thesecond memory location to be transferred on the second data bus.
 65. Thememory controller of claim 64, wherein the logic is configured to changebetween the first mode of operation and the second mode of operationbetween memory accesses.
 66. A memory component comprising, a memorycore for holding read data information; an interface that is coupled toa first data bus and a second data bus; and logic having a first mode ofoperation and a second mode of operation, the first mode of operationcomprising a mode of operation in which a first memory access causes afirst data symbol associated with a first memory location to betransferred on the first data bus and causes a second data symbolassociated with a second memory location to be transferred on the seconddata bus; and the second mode of operation comprising a mode ofoperation in which a second memory access causes the first data symbolassociated with the first memory location to be transferred on the firstdata bus and at a later time a third memory access causes the seconddata symbol associated with the second memory location to be transferredon the second data bus.
 67. The memory component of claim 66, whereinthe logic is configured to change between the first mode of operationand the second mode of operation between memory accesses.